-- Logs begin at Fri 2022-07-01 06:21:41 UTC, end at Fri 2022-07-01 07:40:26 UTC. -- Jul 01 06:54:20.878898 dv-cinder-807215-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 01 06:54:20.886086 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 1 06:54:20 2022] *** Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: nodename: dv-cinder-807215-2-iscsi Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: machine: x86_64 Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: clock source: unix Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: pcre jit disabled Jul 01 06:54:20.888133 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: detected number of CPU cores: 4 Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: current working directory: / Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: detected binary path: /usr/bin/uwsgi-core Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: your processes number limit is 47680 Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: your memory page size is 4096 bytes Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: detected max file descriptor number: 2048 Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: lock engine: pthread robust mutexes Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: thunder lock: enabled Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 01 06:54:20.888637 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 01 06:54:20.934343 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: Python main interpreter initialized at 0x56038c2309c0 Jul 01 06:54:20.934343 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: python threads support enabled Jul 01 06:54:20.934343 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: your server socket listen backlog is limited to 100 connections Jul 01 06:54:20.934343 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: your mercy for graceful operations on workers is 90 seconds Jul 01 06:54:20.934732 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: mapped 403077 bytes (393 KB) for 2 cores Jul 01 06:54:20.934837 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: *** Operational MODE: preforking *** Jul 01 06:54:20.935095 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: *** uWSGI is running in multiple interpreter mode *** Jul 01 06:54:20.935095 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: spawned uWSGI master process (pid: 80131) Jul 01 06:54:20.935295 dv-cinder-807215-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 01 06:54:20.937310 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 1 (pid: 80142, cores: 1) Jul 01 06:54:20.937310 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: spawned uWSGI worker 2 (pid: 80143, cores: 1) Jul 01 06:54:20.937310 dv-cinder-807215-2-iscsi devstack@keystone.service[80131]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 01 06:54:26.384507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 06:54:26.384507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 01 06:54:26.385298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 06:54:26.385298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 01 06:54:26.385707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 06:54:26.385707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 01 06:54:26.385813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 01 06:54:26.385813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 01 06:54:26.434192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 06:54:26.434355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 01 06:54:26.434438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 06:54:26.434525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 01 06:54:26.434597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] command line args: [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 06:54:26.434711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] command line args: [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 01 06:54:26.434848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 06:54:26.435003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 01 06:54:26.435081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ================================================================================ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 06:54:26.435182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ================================================================================ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 01 06:54:26.435261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.435513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_dir = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.435640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.435711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.435903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_source = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.436005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_dir = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.436087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] debug = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.436216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.436785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_source = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] debug = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.437920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_config_append = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_config_append = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_dir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.438968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_dir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.439947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.440948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.441915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.442971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] publish_errors = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] publish_errors = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.443838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] standard_threads = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] standard_threads = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.444917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_journal = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_json = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_journal = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_stderr = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_json = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.445920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_syslog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.446045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_stderr = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.446045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_syslog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.446255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.446371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 01 06:54:26.446489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.446602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.446725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.446854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.446976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.447941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.448882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.449976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.450888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.451932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.452973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.453906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.454954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.455941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.456982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.457915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.458957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.459958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.460877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.461979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.462970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.463893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.464925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.465968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.466931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.467922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.468990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.469930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.470883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.external = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.external = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.471977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.password = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.password = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.token = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.token = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.472914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.473952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.474900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.475971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.476904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.477839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.478937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.479987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.480978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.481901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.482957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.483610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.483729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.483843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.483969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.484893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.485929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.486903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.487951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.488982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.489909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.490976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.491910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.492950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.493996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.494931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.495978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.496905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.497927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.498621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.499416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.499539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.499654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.499768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.499882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.500934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.501966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.502916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.503957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504764 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.504948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.505988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.506938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.507979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.508884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.509910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.510965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.511897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.512939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.513984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.514887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.515945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.516981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.517922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.518993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.519926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.520959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 06:54:26.521888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 01 06:54:26.521888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 01 06:54:26.550667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.550922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.551146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.551232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.551380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.551611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.559819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.560208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.588179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.588365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.588492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.588601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.731111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.731484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.779716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.780473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.780738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.780975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.849532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.849761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.849991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.850219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.850568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.850655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.850836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.850998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.851082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.851276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.851496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.851714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.876729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.876877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.876963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.877938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.878021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.878150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.889043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.889396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.901668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.901904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.902934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.903020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.903100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.903222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.903438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 01 06:54:26.921350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.921562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 06:54:26.921742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.921901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.921987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 06:54:26.922102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 06:54:26.922272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.922356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.922472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 01 06:54:26.922624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.922742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.922915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.923035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.923177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.923292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.923549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.934272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.934555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.934692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.934857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.934994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.935147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.935265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.935543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.943479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.943824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.946784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.947066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.947150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.947354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.947438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.947643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.947737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.947930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.948999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.949800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.976554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.976930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.978167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.978448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.978534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.978816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.984332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.984565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.984711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.984960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.985043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.985373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.992292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.992575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.992659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.992921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.997070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.997363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.997560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:26.997646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.997851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.997947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.998820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.999015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.999100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.999285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:26.999555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.018385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.018563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.018652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.018864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.023268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.023486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.026917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.027102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.027197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.027395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.032906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.033171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.036546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.036791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.040205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.040474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.040559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.040770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.040850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.041904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.061014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.061300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.061414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.061577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.061664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.061842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.061927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.062102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.069769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.070020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.071511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.071743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.071861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.072021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.075133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.075255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.075417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.075533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.075702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.075811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.075986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.076077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.076275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.076359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.090077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.090173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.090356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.090444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.090635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.090719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.090941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.091027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.091224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.091307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.093299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.093390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.093600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.093685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.111528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.111619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.114791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.114895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.118082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.118166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.119845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.119930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.121473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.121559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.121757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.121838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.122034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.122114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.129473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.129560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.131100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.131184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.134335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.134430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.134634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.134716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.135058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.135141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.135345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.135430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.144984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.145077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.145268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.145353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.145546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.145629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.153427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.153514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 01 06:54:27.153710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.153806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.153991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.154987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.155573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.155782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.155865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.156974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.157780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 01 06:54:27.195860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56038c2309c0 pid: 80143 (default app) Jul 01 06:54:27.195995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56038c2309c0 pid: 80142 (default app) Jul 01 06:54:27.199298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad62a1-df37-4147-bf73-d7c5aff16ba1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:27.199593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad62a1-df37-4147-bf73-d7c5aff16ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:27.199874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad62a1-df37-4147-bf73-d7c5aff16ba1 None None] PATH_INFO: `/v3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:27.204634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Fri Jul 1 06:54:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:34.838978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:34.839577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:34.839577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:34.844517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:54:34.849791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:34.850217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:34.850356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:34.923488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: INFO dbcounter [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] Registered counter for database keystone Jul 01 06:54:34.945358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG oslo_db.sqlalchemy.engines [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 06:54:34.951464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writer thread running {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 06:54:35.128884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80142) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 06:54:35.129305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80142) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 06:54:35.183065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:35.184672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:35.306271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 06:54:35.485518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:35.558562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: INFO dbcounter [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] Registered counter for database keystone Jul 01 06:54:35.578232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG oslo_db.sqlalchemy.engines [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 01 06:54:35.584391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writer thread running {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 01 06:54:35.707387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acdb2d-e3c9-449f-97c3-61e3b7768800 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:35.711688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:35.712348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:35.712637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:35.712914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:35.721951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:35.783176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 06:54:35.783444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c640649-192c-4198-9e27-b20d6278a9b8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:35.796386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Fri Jul 1 06:54:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 06:54:35.802288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:35.818047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee66907-1524-48b9-9cc1-b9fa4dd4e617 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:35.822410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:35.823556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:35.823841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:35.824114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] PATH_INFO: `/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:35.825618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] RBAC: Authorizing `identity:get_project(project_id=b0142643ea3d4627a8dc9edbdfab9d31)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:35.880353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING keystone.common.rbac_enforcer.enforcer [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 01 06:54:35.880958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b90e41-360d-4187-9f91-5008ed821968 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:35.885369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Fri Jul 1 06:54:35 2022] GET /identity/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:37.909170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bb5d8-e90a-4f79-9090-388bf5247773 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:37.909706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bb5d8-e90a-4f79-9090-388bf5247773 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:37.909706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bb5d8-e90a-4f79-9090-388bf5247773 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:37.913823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:54:37.919285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:37.919672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:37.919850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:37.939086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:37.940613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:37.948422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 01 06:54:37.970711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bb5d8-e90a-4f79-9090-388bf5247773 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:37.982480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bb5d8-e90a-4f79-9090-388bf5247773 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:37.987327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:37.987939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:37.988219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:37.988493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:37.996005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:37.998076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342dc4df-7d46-44ce-b7dc-5840e1b2fe46 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:38.006723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Fri Jul 1 06:54:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 06:54:38.011801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:38.020619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc384ad-6dc7-485c-878f-fdc501514e55 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:38.024945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:38.025790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:38.026072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:38.026345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:38.027763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:38.029585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbdb4a-787e-4e58-8c88-6fc74416455f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:38.031678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Fri Jul 1 06:54:38 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:40.089506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379822-5c1d-4821-a5b3-8f53d0cda3dc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.090018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379822-5c1d-4821-a5b3-8f53d0cda3dc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.090100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea379822-5c1d-4821-a5b3-8f53d0cda3dc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.091454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:40.096595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.096996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.097152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.113783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:40.114988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:40.121327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174430-0d3a-4ca0-af80-11d0f3043c59 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.121617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174430-0d3a-4ca0-af80-11d0f3043c59 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.121895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e174430-0d3a-4ca0-af80-11d0f3043c59 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.123272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:54:40.128886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.129174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.129452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.168926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80143) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 01 06:54:40.169354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80143) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 01 06:54:40.198916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:40.221712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.227226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:40.228824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:40.246728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.328537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:54:40.331374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2e1b-8cbb-474e-953f-02ff3e958eef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.351464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.365192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.365995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.366283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.366565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.377561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:40.388733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:40.389703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:54:40.390545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:54:40.395592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.449823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d723ace-19a8-4c14-9587-5fb3f427ff9a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.452154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.476421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.487790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.488682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.488970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.489245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.495466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3974e8-15c0-41e8-ab19-a55beafcacee None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.529301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.529962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.530802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.531086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.531832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:40.532137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.533528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.545072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:40.550439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.556667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.558176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.648986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0242f1-0295-4fc7-b443-d8fc38c7f5aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.654433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:54:40.660450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.663052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.684205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.685075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.685364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.685644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.686880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.688342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.690129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:40.695233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.710371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.757268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc31af-a560-4004-ba24-69849cf27a01 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.791203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.791984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.792267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.792539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.793778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5d2be-c514-4bc9-bc22-dcf6feca1373 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.812227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.813674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.826172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.827480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.827761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.828039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.829214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.830638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.832456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:54:40.834356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:40.835236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:40.836036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:54:40.836036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:54:40.836036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:54:40.836036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:40.836036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:54:40.837950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.841026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:40.896407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.918695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:40.937831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484f711-76c5-4745-97ce-91de8a15106d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.946845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543c3e3-31bc-4abb-b608-4ba0025a8cd2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:40.972419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.973081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.973363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.973639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.974926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.domain.id=default, user.id=bb77d87b954348508ac808b62617ddb0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:40.975242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.976671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3283e-9e33-460e-a8b6-e5ac6243e660 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:40.981101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:40.981767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:40.982056 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:40.982340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:40.983110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:40.983454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:40.983645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 11/20] 10.222.0.48 () {56 vars in 1322 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/role_assignments?user.id=bb77d87b954348508ac808b62617ddb0&scope.domain.id=default&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:40.984842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df3c3e-184f-4075-9c6b-31d385dad133 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:41.004593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 10/21] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:43.172527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45a71a-4951-4eda-ab28-b4f89ab71a0f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.173059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45a71a-4951-4eda-ab28-b4f89ab71a0f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.173358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45a71a-4951-4eda-ab28-b4f89ab71a0f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.174858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 12/22] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:54:43.180315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.180980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.181156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.208566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:43.209778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:43.295674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 11/23] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:43.337142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45a71a-4951-4eda-ab28-b4f89ab71a0f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:43.485039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45a71a-4951-4eda-ab28-b4f89ab71a0f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:43.532944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:43.533791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.534077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.534356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.554958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-df194c60-60cf-4a11-835e-1e3a932b60c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:43.563740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:43.564782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:54:43.565636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 13/24] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:43.573016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:43.611971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775cc4-6cb4-4155-9c17-fcc597856f9b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.612259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775cc4-6cb4-4155-9c17-fcc597856f9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.612699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775cc4-6cb4-4155-9c17-fcc597856f9b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.614637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 14/25] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:43.627266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.627750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.628184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.643147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:43.649945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:43.657208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:43.658492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:43.747992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9514f8e-91e4-465c-89a6-d98869c42619 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:43.801227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 15/26] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:43.808368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:43.809048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:43.809329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:43.809606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:43.810366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:43.810670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:43.812657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:43.834908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 12/27] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:43.840754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:43.844762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:43.998402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f5fc-4b81-489b-a4c1-4e5a4997c32c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.045501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963238d-93a8-4f4c-a41a-d4bb860578c3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.050648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.052506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.053321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.054805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.057042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.059306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.061754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eae64-07f8-4729-9ca7-55039e8590c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.063500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 16/28] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:43 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:44.064766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44f85-28d3-40dd-a97a-8ad9720bc279 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.065252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44f85-28d3-40dd-a97a-8ad9720bc279 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.065709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44f85-28d3-40dd-a97a-8ad9720bc279 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.067129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 17/29] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:44.068351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef166a-91fd-41a8-bd1d-eb0a819e2a2d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.068816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef166a-91fd-41a8-bd1d-eb0a819e2a2d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.069253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef166a-91fd-41a8-bd1d-eb0a819e2a2d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.074523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 18/30] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:44.075502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef166a-91fd-41a8-bd1d-eb0a819e2a2d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.078824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.094324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.095241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.095636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.096019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.107690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.109844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0010f3b2-268a-4418-9725-2f9ad1396835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:54:44.129530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:44.130337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:44.131300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:54:44.131300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:54:44.131300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:54:44.131300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:44.131300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 13/31] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:54:44.136143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a30d3-e238-4943-b6e1-44c6e36e79ec None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.136143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a30d3-e238-4943-b6e1-44c6e36e79ec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.136143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a30d3-e238-4943-b6e1-44c6e36e79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.151665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-026a30d3-e238-4943-b6e1-44c6e36e79ec None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:44.153020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-026a30d3-e238-4943-b6e1-44c6e36e79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:44.212834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef166a-91fd-41a8-bd1d-eb0a819e2a2d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.239192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 14/32] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:44.241244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.241657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.242085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.276303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:44.277405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:44.281253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.282050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.282330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.282602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.284710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.285976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.287870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 19/33] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:44.288391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.364555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.364955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 15/34] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:44.366118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.402886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cce7f-871a-4fde-a9a3-862e14ec8af2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.403252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.437473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.438168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.438526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.438833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.439570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:44.439880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.441366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.460600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 20/35] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:44.461361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.496404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99413e-5dd2-49cc-baac-1b6a7d80b378 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.532845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.533444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.533729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.534007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.534759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.536226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.576367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bed1e4-006b-42b3-bb70-0544521e76d7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.612345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.619089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.619989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.620279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.620565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] PATH_INFO: `/v3/domains/default/users/bb77d87b954348508ac808b62617ddb0/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.623565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bb77d87b954348508ac808b62617ddb0, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.625591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.677783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 16/36] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/roles => generated 219 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:44.684057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.750536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 21/37] 10.222.0.48 () {58 vars in 1361 bytes} [Fri Jul 1 06:54:44 2022] PUT /identity/v3/domains/default/users/bb77d87b954348508ac808b62617ddb0/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:54:44.758973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:44.816894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:44.864684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681ec3f-c1a1-472a-8073-f1acd3abf268 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:44.912370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:44.912961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.913258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.913549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.914569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:44.916644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30bc1f-f0c5-4fb3-bc69-42bd0296d5b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:44.942084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 17/38] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/roles => generated 225 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:44.943712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea46326-aeb5-41ae-94b4-1272ba7f0981 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.944013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea46326-aeb5-41ae-94b4-1272ba7f0981 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.944303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea46326-aeb5-41ae-94b4-1272ba7f0981 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.946494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 18/39] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:44.950801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e18a-36a9-4f5c-a5ea-36ff1fdef4cb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.950801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e18a-36a9-4f5c-a5ea-36ff1fdef4cb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.950801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e18a-36a9-4f5c-a5ea-36ff1fdef4cb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.950801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 19/40] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:44.957058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65e92-f34e-4ef2-a7fa-5d9c3a5345e5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:44.957488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65e92-f34e-4ef2-a7fa-5d9c3a5345e5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:44.957783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c65e92-f34e-4ef2-a7fa-5d9c3a5345e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:44.989117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-74c65e92-f34e-4ef2-a7fa-5d9c3a5345e5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:44.990829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-74c65e92-f34e-4ef2-a7fa-5d9c3a5345e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:45.065531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dd2b7-353b-481c-b6a3-1d870744a3e2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:45.076511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 20/41] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:45.078039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:45.078334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:45.078712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:45.096172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:45.097393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:45.103640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:45.104332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:45.104622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:45.104907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:45.105639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:45.107401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:45.153125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 22/42] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:54:44 2022] POST /identity/v3/projects => generated 301 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:45.154816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:45.197116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:45.205862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 21/43] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:45.241018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:45.243932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:45.334153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c2023-2827-420c-8c06-014b99b13575 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:45.376188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0bb31-4dfe-4910-9065-ad6718bd7326 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:45.377793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:45.378677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:45.379021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:45.379298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:45.381124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:45.382032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:45.383897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 23/44] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:45.389270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:45.421767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:45.422918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:45.423400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:45.423689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:45.424457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:45.425871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70deba2b-a1be-4ccb-aca7-297184772823 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:45.442013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 22/45] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Jul 1 06:54:45 2022] POST /identity/v3/roles => generated 223 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:54:45.512766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc0092-0135-4417-b3fb-3744ffadeb3d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:45.527152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:45.553471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:45.554159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:45.554442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:45.554719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:45.555492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:45.557256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1ca9-01d8-4e0c-997c-17abb1704192 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:45.585918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 24/46] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:54:45 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:54:46.919716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985d93-0ba8-404d-b41a-b12b9929d881 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:46.920239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985d93-0ba8-404d-b41a-b12b9929d881 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:46.920323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985d93-0ba8-404d-b41a-b12b9929d881 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:46.921665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 23/47] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:46.926922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:46.927319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:46.927603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:46.945012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:46.946520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:47.011492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 25/48] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:47.034394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985d93-0ba8-404d-b41a-b12b9929d881 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.092963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:47.134076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985d93-0ba8-404d-b41a-b12b9929d881 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.166292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.167115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.167408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.167686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.179110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:47.188515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:47.189354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:47.190175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 24/49] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:47.197492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.286553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:47.296345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e44a2-e67d-4e1b-9116-384ceaa93ac7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.330257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.330958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.331248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.331530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.332283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:47.332592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:47.333962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:47.341919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 26/50] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:47.347533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.445916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682041b-ebd9-4002-9720-18235755ad03 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.464705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:47.478863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.479741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.480028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.480315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.481909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:47.483939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bd19d-2b1a-4460-97e6-e398477f9c0b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:47.485581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 25/51] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:47.491050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.587201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d135c-c99f-4972-a109-e1b32ee21720 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.619114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.619901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.620200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.620500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.621670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:47.621989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d372-af5c-499a-a0ff-ebeb6cfff86d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.622286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d372-af5c-499a-a0ff-ebeb6cfff86d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.622573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214d372-af5c-499a-a0ff-ebeb6cfff86d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.623138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:47.624568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 26/52] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:54:47.625291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 27/53] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:47.629585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.629974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.630264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.630705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.656060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:47.734935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4c04-ca88-4cd9-a7f3-0fc191e107b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.750160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:47.751406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:47.767954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.768609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.768900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.769184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.770019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.domain.id=default, user.id=bb77d87b954348508ac808b62617ddb0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:47.770335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:47.771819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:47.777593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 28/54] 10.222.0.48 () {56 vars in 1322 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/role_assignments?user.id=bb77d87b954348508ac808b62617ddb0&scope.domain.id=default&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:54:47.821867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 27/55] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:47.845906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:47.936448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:47.952846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cec0de-60b8-4d81-857e-64c4216e79a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:47.986849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:47.987576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:47.987878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:47.988149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:47.989322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:47.990852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ab60b-3f79-46b9-9233-5b4d6c393d4c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:47.992492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 29/56] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:47.998114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:48.094779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5d365-0929-4bf1-8a27-18cae04b78dc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:48.111184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:48.127773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:48.128428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:48.128718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:48.129001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:48.129720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:48.131213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d7a8b-0c39-42d5-a39f-d587512f3bbe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:48.157112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 28/57] 10.222.0.48 () {60 vars in 1140 bytes} [Fri Jul 1 06:54:47 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:50.177677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057db89-a3db-491e-8a08-961ed929b17e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:50.178198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057db89-a3db-491e-8a08-961ed929b17e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:50.178281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057db89-a3db-491e-8a08-961ed929b17e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:50.179642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 30/58] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:50.184873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:50.185234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:50.185442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:50.203189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:50.204362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:50.269128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 29/59] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:50.293078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057db89-a3db-491e-8a08-961ed929b17e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:50.395256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057db89-a3db-491e-8a08-961ed929b17e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:50.426675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:50.427475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:50.427804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:50.428036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:50.429196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:50.430617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fddca2-8c7c-4a29-b163-1ea5e62cbf05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:50.432476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 31/60] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:50.438474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:50.529689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:50.532590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf05095-b9d6-46d3-91eb-6f65a9cedf41 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:50.566254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:50.566941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:50.567228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:50.567506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:50.568219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:50.569649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04253b03-1a39-4133-ae94-a44d29c4a513 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:50.595291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 30/61] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:54:50 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:53.637376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c19b2-d3bc-4c59-ae22-f0a30fdef8bc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:53.638355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c19b2-d3bc-4c59-ae22-f0a30fdef8bc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:53.638891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c19b2-d3bc-4c59-ae22-f0a30fdef8bc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:53.640525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 32/62] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:53.645736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb9acb-8d46-448f-8383-59086a30a5a8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:53.646515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb9acb-8d46-448f-8383-59086a30a5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:53.647022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb9acb-8d46-448f-8383-59086a30a5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:53.672282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5acb9acb-8d46-448f-8383-59086a30a5a8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:53.673841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5acb9acb-8d46-448f-8383-59086a30a5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:53.756331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 31/63] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:53.792113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c19b2-d3bc-4c59-ae22-f0a30fdef8bc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:53.832475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:53.889174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63e0d-046a-4d66-b34b-33181798fd20 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:53.889831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63e0d-046a-4d66-b34b-33181798fd20 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:53.890316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63e0d-046a-4d66-b34b-33181798fd20 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:53.891923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 32/64] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:53.897149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da035-4007-4f20-ad61-5d12a56ef0d9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:53.897683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da035-4007-4f20-ad61-5d12a56ef0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:53.898176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da035-4007-4f20-ad61-5d12a56ef0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:53.934087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d4da035-4007-4f20-ad61-5d12a56ef0d9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:53.936882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d4da035-4007-4f20-ad61-5d12a56ef0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:53.956398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c19b2-d3bc-4c59-ae22-f0a30fdef8bc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.003926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.004998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.005525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.005945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.018253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:54.023741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 33/65] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:54:54.030789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70cf36-3bc6-4437-95fc-d77e35e4b2ff None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e5fde1d7-3eb8-4c17-9319-1a854c43c648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:54.030911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:54.031700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:54:54.032533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 33/66] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:54.033151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70cf36-3bc6-4437-95fc-d77e35e4b2ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.033151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70cf36-3bc6-4437-95fc-d77e35e4b2ff None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.033151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 34/67] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:54.038266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957262b-9862-432a-a933-decf23fb6119 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.038266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957262b-9862-432a-a933-decf23fb6119 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.038266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957262b-9862-432a-a933-decf23fb6119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.040038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70cf36-3bc6-4437-95fc-d77e35e4b2ff None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.064205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3957262b-9862-432a-a933-decf23fb6119 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:54.066363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3957262b-9862-432a-a933-decf23fb6119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:54.067147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.161432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 34/68] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:54:54.162870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957262b-9862-432a-a933-decf23fb6119 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.181299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70cf36-3bc6-4437-95fc-d77e35e4b2ff None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.188225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.240228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.241433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.241958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.242461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.243977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:54.244492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.246335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.255131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 35/69] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:54.256112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.299151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957262b-9862-432a-a933-decf23fb6119 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.334998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.335798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.336158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.336441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.338709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.347518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f35a3d33-ccbd-4d2b-a862-bb28ef9a7cf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:54.355022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:54.355879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:54:54.356630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 35/70] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:54.357407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848965d-a7bc-4a23-b152-1ff32da6e1eb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.357761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848965d-a7bc-4a23-b152-1ff32da6e1eb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.357932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848965d-a7bc-4a23-b152-1ff32da6e1eb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.359242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 36/71] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:54.359938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848965d-a7bc-4a23-b152-1ff32da6e1eb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.364835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18687c8f-9e10-4513-ac59-586efc2fbb73 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.398640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.399467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.399753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.400032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.410162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-769687a0-5b2f-496a-b64d-2d506c0da891 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:54.418129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:54.418976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:54.419738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 36/72] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:54.420470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee685-648a-4c21-b7b5-90e7250fceeb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.420835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee685-648a-4c21-b7b5-90e7250fceeb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.421053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ee685-648a-4c21-b7b5-90e7250fceeb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.422345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 37/73] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:54.423054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ee685-648a-4c21-b7b5-90e7250fceeb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.433651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.465118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848965d-a7bc-4a23-b152-1ff32da6e1eb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.502846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.503715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.503999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.504304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.517925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.519431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1f96c-5093-4cc8-b924-7d7f1b605f5e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.521088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 37/74] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:54.522248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdeb9d-aa72-4aff-87c2-bf8a9e23e65a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.522537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdeb9d-aa72-4aff-87c2-bf8a9e23e65a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.522904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdeb9d-aa72-4aff-87c2-bf8a9e23e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.524376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ee685-648a-4c21-b7b5-90e7250fceeb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.538680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-36fdeb9d-aa72-4aff-87c2-bf8a9e23e65a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:54.539943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36fdeb9d-aa72-4aff-87c2-bf8a9e23e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:54.557565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.558230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.558517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.558824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.559572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:54.559878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.561244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.566533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.570795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 38/75] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:54.571577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.612094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 38/76] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:54.613424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.613717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.613995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.629635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:54.630841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:54.670957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a57ae-cb93-48e1-b543-f663fad6ab5f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.688311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.699772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 39/77] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:54.700586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.704609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.705277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.705559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.705835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.706578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:54.706924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.708299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.716305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 39/78] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:54.717196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.777145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.814721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3420cb-4c25-4193-a7db-a6a78b25ba9b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.816154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d18154-b989-406e-92d4-875fa1e47020 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.847411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.847855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.848063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.848356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.848641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.848728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.848994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.849273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.850469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.851980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.853609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 40/79] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:54.854376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:54.905757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.916409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:54.917880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:54.950835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749aefd-0dc8-48fc-9ba3-f7b3c57b3aad None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:54.977948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:54.984335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:54.985119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:54.985405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:54.985687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:54.995600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:55.007970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:55.009228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 41/80] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:55.010158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.016597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 40/81] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:54:55.017429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.107068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.112262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2720-28ee-461f-81ad-8dc110cbbbe9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.117330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567f136-fb1a-4172-916e-ea8a70747ede None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.144916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.145781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.146067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.146351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.147587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.149024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.150082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.150651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 42/82] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.150899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.151185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.151464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.152633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.154078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.155632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.160488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 41/83] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.160488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.217347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.255776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d90d8-8196-4fd0-b2e5-45ab0314e001 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.258938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b673200-2556-4abc-9b19-d953be40646e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.289059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.289729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.290009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.290291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.290845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.291071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:55.291382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.291489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.291775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.292054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.292757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.301194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 42/84] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.301976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.322926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.359035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.360493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.398569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7efb6-134a-4ab9-a4fe-3eee9b1735c9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.414321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.431894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.432751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.433033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.433309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.434492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.435955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.437725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 43/85] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.442673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.456374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 43/86] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:54:55.457187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.526385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.539058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a2e82-17fb-4da2-aa33-26c56359b569 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.553468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb51bdf-121f-4a80-a638-f3701d2fa5eb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.570957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.571598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.571885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.572168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.591537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.593333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.593785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.594069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.594828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.596299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.634799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 44/87] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:54:55 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:55.634799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.642347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.644408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.645930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.737196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a807a4-2eb6-4da0-9eaf-2ae3d465c9be None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.761509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.773103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.774237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.774323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.774543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.775801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.777318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.779372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 45/88] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.779714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.781781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 44/89] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:54:55.782669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.889189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:55.901744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24363e-8cc4-477a-a982-6e544d019b64 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.907815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdae0d-f0f4-4906-85ca-7d005ec61136 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:55.949033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.949240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:55.949692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.949974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.950253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.950743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:55.951135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=9b03423a023143208d210010093cfe64) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:55.951341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:55.951444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.951860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:55.953427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.953922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:55.955747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:55.957638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 45/90] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.958572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:55.962507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 46/91] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=41985665942d4bba9f8355bf503f068b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:55.966247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:56.031259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:56.082607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049ad49-4868-462a-af2b-634681b9e4c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.093066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4cf9-e3e2-4bb0-9b42-61cdb0430d95 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.120812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.121472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.121762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.122043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.132397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.133587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.134095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.134574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.136068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.137750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.140387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 47/92] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:56.141391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:56.158588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:56.197632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.199168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.259235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:56.273633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88032a8d-b4d0-4f00-91c7-67af427498bf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.307835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 46/93] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:54:56.308631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:56.325040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.326037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.326560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.327101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.331797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=bb77d87b954348508ac808b62617ddb0) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:56.332378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.334079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.346819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 48/94] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:54:56 2022] GET /identity/v3/role_assignments?user.id=bb77d87b954348508ac808b62617ddb0&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:56.348308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:56.428923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8fd0-3bc0-4e31-b672-44b1c9e9a300 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.488297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.489091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.489387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.489673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.490531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=9b03423a023143208d210010093cfe64) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:56.490886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.492369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.499755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 47/95] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:54:56 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=1dfd6d1514104a199766879261c434f7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:56.502665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:56.546934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696731eb-cfe5-47cc-8cd7-a767fb79f862 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.613817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.614698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.615006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.615281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] PATH_INFO: `/v3/projects/d1ccba3669b84275b82a48cb63c5dabe` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.619711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ccba3669b84275b82a48cb63c5dabe)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.621206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8552cef-0a03-43cc-bebd-7a35d6463959 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.623416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 49/96] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:54:56 2022] GET /identity/v3/projects/d1ccba3669b84275b82a48cb63c5dabe => generated 312 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:56.629626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:56.664912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:56.772613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d14347-4b8d-467c-a182-3fe50880647f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:56.822808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:56.823474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:56.823762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:56.824047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:56.824885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=d1ccba3669b84275b82a48cb63c5dabe, user.id=9b03423a023143208d210010093cfe64) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:56.825205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:56.826649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98644a38-c79d-4397-8832-e0048bcfd07f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:56.839398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 48/97] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:54:56 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=d1ccba3669b84275b82a48cb63c5dabe&role.id=41985665942d4bba9f8355bf503f068b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:58.636213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4a4f7-2277-41d5-a8a6-30d99ae22437 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:58.636718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4a4f7-2277-41d5-a8a6-30d99ae22437 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:58.636792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4a4f7-2277-41d5-a8a6-30d99ae22437 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:58.638159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 50/98] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:58.643339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:58.643757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:58.643921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:58.660441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:58.661640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:58.729228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 49/99] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:54:58.753466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4a4f7-2277-41d5-a8a6-30d99ae22437 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:58.858741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4a4f7-2277-41d5-a8a6-30d99ae22437 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:58.893647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:58.894370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:58.894657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:58.894965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:58.906075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eccfa409-7288-4d5c-89da-108c16daaae2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:58.913803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:58.914670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:58.915458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 51/100] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:58.925174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:58.941701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533a325-f48a-40a9-8784-25c04a8563f2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:58.942027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533a325-f48a-40a9-8784-25c04a8563f2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:58.942279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533a325-f48a-40a9-8784-25c04a8563f2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:58.943835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 52/101] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:58.954348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:58.960863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5b4a-ee83-4124-adaf-2bc452dab6a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:58.961164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5b4a-ee83-4124-adaf-2bc452dab6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:58.961482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5b4a-ee83-4124-adaf-2bc452dab6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:58.985428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.005374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6d0a5b4a-ee83-4124-adaf-2bc452dab6a7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:59.007199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d0a5b4a-ee83-4124-adaf-2bc452dab6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:59.056543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c597b3-0080-433f-8ba0-f9c6026f2915 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.104269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.104960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.105254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.105522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.106446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:59.106601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.108022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.126734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 53/102] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:54:59.128285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657f2c-5ce3-4747-a2c4-824ad98459b5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.128662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657f2c-5ce3-4747-a2c4-824ad98459b5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.129017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31657f2c-5ce3-4747-a2c4-824ad98459b5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.130347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 54/103] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:59.135689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.136043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.136530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.136631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 50/104] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:54:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:59.142208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.170060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:59.173133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:59.274584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.302970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 55/105] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:54:59.303919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.311647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413cf5d-dd72-45e3-8861-9b2444d43c81 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.381346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.384470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.384972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.385468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] PATH_INFO: `/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.415309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] RBAC: Authorizing `identity:get_user(user_id=46dd9c1766844b92b4c8b8eeecf2ee3f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.418402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.421867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 51/106] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f => generated 289 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:59.426839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.450411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.497919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f21e5-0fb1-491e-982b-02371853e080 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.548716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.549522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.549813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.550096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.552403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4266-4aa6-48dd-8825-f15be09820ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.567468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:54:59.579105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:54:59.579979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:54:59.580741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:54:59.583041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 56/107] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:54:59.584599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.593385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.594308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.594699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.595124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.596429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.598085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2c655-0ec3-40fd-82fd-9034b6ed382c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.600516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 52/108] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:59.602067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654ff1-2f8a-42ce-915b-d53927485a90 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.602429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654ff1-2f8a-42ce-915b-d53927485a90 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.602826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654ff1-2f8a-42ce-915b-d53927485a90 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.609243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 53/109] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:59.610147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff654ff1-2f8a-42ce-915b-d53927485a90 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.613375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.713314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681af0c-4e99-470e-aca0-58d50ed8476b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.728738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff654ff1-2f8a-42ce-915b-d53927485a90 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.741358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.750610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.751288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.751579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.751863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.763013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.763683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.763971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.764254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.765009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:54:59.765329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.766724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.774634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 54/110] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:54:59.775430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.818986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.820446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bbb94-f1a4-4365-85e9-2d1624e04f01 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.839599 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:54:59.879291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46452e-6cd0-48de-8d2f-a7c49b44ceaa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:54:59.911854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:54:59.912512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.912799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.913085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.913790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:54:59.915274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e6510-1115-489b-9007-52f7ec866f91 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:54:59.918450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 57/111] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:54:59.919742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.920035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.920312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.937468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:54:59.938809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:54:59.944140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 55/112] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:54:59 2022] POST /identity/v3/projects => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:54:59.945582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ff84-fd65-4519-b180-0938dbf1dc60 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.945887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ff84-fd65-4519-b180-0938dbf1dc60 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.946171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1ff84-fd65-4519-b180-0938dbf1dc60 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 56/113] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824991-3de5-4f2d-a054-e5fe950f8003 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824991-3de5-4f2d-a054-e5fe950f8003 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9824991-3de5-4f2d-a054-e5fe950f8003 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 57/114] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:54:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:54:59.951281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824991-3de5-4f2d-a054-e5fe950f8003 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:54:59.973347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.032081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 58/115] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:00.032967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.103271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9824991-3de5-4f2d-a054-e5fe950f8003 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.123287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.145915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.146811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.147103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.147410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.148606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.150101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211382b4-73e9-4ee1-a901-9812deb22fc2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.151799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 58/116] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:54:59 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:00.157096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.157096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.157096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.169433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09b4fe-51b9-43bb-8da6-f8db75a94bae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.172009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:00.173257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:00.210402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.211327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.211643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.211926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.213134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.214589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88442445-482d-44e6-a985-fb8df199c1d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.216341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 59/117] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:00.217505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.218348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.218629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.235649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:00.236678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:00.253528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 59/118] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:00.254499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.261192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.329979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 60/119] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:00.330926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.366054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca814c-675c-4866-8be1-a2b4a7267b54 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.382186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.403417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.404603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.404893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.405221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.415312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:00.423002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:00.423859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:00.424622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 60/120] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:00.425267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.453654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998903e-4e1b-4c75-b274-6557588c626c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.488958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.489746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.490038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.490321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.491535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.492984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.494687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 61/121] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:00.495474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.522253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.538756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7464e5b-fe87-4945-8ee3-a186759834c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.574855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.575503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.575786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.576062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.576887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=46dd9c1766844b92b4c8b8eeecf2ee3f) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:00.577747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.579224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.585073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 61/122] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/role_assignments?user.id=46dd9c1766844b92b4c8b8eeecf2ee3f&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=96d3a4c0e9534751998d64755241c000 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:00.586261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.607162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecc8dd-c925-4c67-9f04-a7dddc2f506c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.661319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.665332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.666281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.666576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.666895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.683685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:00.694871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:00.695870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:00.696689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 62/123] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:00.697371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.737621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e5c89-98db-438a-90e8-6e471a199645 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.782702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.783578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.783938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.784295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.795035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:00.803631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:00.804504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:00.805267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:00.810148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 62/124] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:00.811123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.819149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.830299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943ae2-1572-4f96-8080-9016f63cdcfb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.872515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.873734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.875055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.875529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.877016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:00.877517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.879860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.890401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 63/125] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:00.891524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:00.923467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:00.927437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc68ffd-5b10-4cad-9df5-1288606574a0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:00.969407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:00.970417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:00.970793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:00.971157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/users/9b03423a023143208d210010093cfe64/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:00.973282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, user_id=9b03423a023143208d210010093cfe64, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:00.974862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:00.992208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 63/126] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:00 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/users/9b03423a023143208d210010093cfe64/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:00.992892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.013233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b36c-9b76-45e2-9a01-67fb549e3609 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.054038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.055147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.055756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.056516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.057602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:01.058120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.059774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.070348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 64/127] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.075921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.095142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:01.117816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de3cc4-59d2-4c24-a5c0-5d5561b525a7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.158528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.160284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.160568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.165635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.166553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:01.167509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.169060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.185866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 64/128] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.186603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.235201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:01.257185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc316459-5a42-4198-a0a1-7b8071ab83db None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.297364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.298399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.298689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.299021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.300232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.301787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.303727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 65/129] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.304541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.319494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacaabd-4764-4cf4-b3a6-f1c8144ab03f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.363543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.364718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.365016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.365303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.366529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.368759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.371476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 65/130] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.372364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.390684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:01.459424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc5b39-1284-4874-b576-5ce8e3250240 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.479177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:01.487855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e51e3-d0c2-425c-9717-de9fb0de6247 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.516635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.517537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.517828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.518110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.519365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.520986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.522823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 66/131] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.523594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.630196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676df07-cc06-496d-8576-f9dc43fa2cde None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.663501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.664417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.664691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.664975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] PATH_INFO: `/v3/projects/d1ccba3669b84275b82a48cb63c5dabe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.667512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] RBAC: Authorizing `identity:get_project(project_id=d1ccba3669b84275b82a48cb63c5dabe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.668322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.669401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.671164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.671653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.672135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.674641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 66/132] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/projects/d1ccba3669b84275b82a48cb63c5dabe => generated 312 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.675540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.677197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.678184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.681343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 67/133] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.688267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.714078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:01.859509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895cffa-19ad-4752-a39e-48fee95e9461 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.888391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa1708-b500-4ae2-b967-d7aaa418f46a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:01.932866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.933768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.934054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.934335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] PATH_INFO: `/v3/projects/d1ccba3669b84275b82a48cb63c5dabe/users/9b03423a023143208d210010093cfe64/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.937550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1ccba3669b84275b82a48cb63c5dabe, user_id=9b03423a023143208d210010093cfe64, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.939065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.945451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:01.946332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:01.946615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:01.946938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:01.950194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:01.951720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:01.954325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 67/134] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:01 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:01.956162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:01.974479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 68/135] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:01 2022] PUT /identity/v3/projects/d1ccba3669b84275b82a48cb63c5dabe/users/9b03423a023143208d210010093cfe64/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:01.975529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:02.007956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:02.092108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:02.155030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ce70f-391f-4109-9dd6-8f9e2a204599 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:02.198728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a2524-f09a-4f0d-a855-b243a9a8b863 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:02.232793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:02.234268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.234553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.234969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/users/bb77d87b954348508ac808b62617ddb0/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.239829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, user_id=bb77d87b954348508ac808b62617ddb0, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:02.241300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5765dc4-bc94-498b-ad5f-4b76683bc962 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:02.245065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:02.246122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.246561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.247045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/users/9b03423a023143208d210010093cfe64/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.250657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, user_id=9b03423a023143208d210010093cfe64, role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:02.252355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f24c36-7e4a-4d5a-be26-fdf4d9d29811 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:02.268691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 68/136] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:01 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/users/9b03423a023143208d210010093cfe64/roles/1dfd6d1514104a199766879261c434f7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:02.269747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 69/137] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:01 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/users/bb77d87b954348508ac808b62617ddb0/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:02.321728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab93f2-f08c-4aa1-aef2-0f634fa5013f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.322062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab93f2-f08c-4aa1-aef2-0f634fa5013f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.322405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab93f2-f08c-4aa1-aef2-0f634fa5013f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.323992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 69/138] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:02.335732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.336273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.336748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.364573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:02.366060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:02.469027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 70/139] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:02.501548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab93f2-f08c-4aa1-aef2-0f634fa5013f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:02.599609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:02.619531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab93f2-f08c-4aa1-aef2-0f634fa5013f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:02.656558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:02.657415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.657698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.658019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.659216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:02.660744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e3a2a-f172-42df-8bf0-0acb2b28f4ec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:02.662759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 70/140] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:02.668099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:02.753788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:02.802845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0f21a-1af5-4f42-b682-b8c2d5a33a2f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:02.848117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:02.849160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:02.849643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:02.850113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:02.851110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:02.852765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a06d3-0f02-486d-8e4e-70818d527fba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:02.891553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 71/141] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:02 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:03.493036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeeccf-c188-4edf-8c3d-640a3706c34a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:03.493508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeeccf-c188-4edf-8c3d-640a3706c34a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:03.493508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeeccf-c188-4edf-8c3d-640a3706c34a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:03.495039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 71/142] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:03.501076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c53bf-80dc-44bc-8455-3da04c3e0713 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:03.501374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c53bf-80dc-44bc-8455-3da04c3e0713 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:03.501656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c53bf-80dc-44bc-8455-3da04c3e0713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:03.523054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4d0c53bf-80dc-44bc-8455-3da04c3e0713 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:03.524238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4d0c53bf-80dc-44bc-8455-3da04c3e0713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:03.606494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 72/143] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:03.637868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a8f-6749-4c19-aedb-5d0dfca7dc8e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:03.638178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a8f-6749-4c19-aedb-5d0dfca7dc8e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:03.638420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a8f-6749-4c19-aedb-5d0dfca7dc8e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:03.639885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 72/144] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:03.645289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8a8f-6749-4c19-aedb-5d0dfca7dc8e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:03.649327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:03.649617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:03.649897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:03.677604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:03.678894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:03.746513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:03.757011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 73/145] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:03.787569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:03.863066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8a8f-6749-4c19-aedb-5d0dfca7dc8e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:03.914928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:03.916079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:03.916556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:03.917021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:03.932815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:03.942808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:03.943935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:03.944679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:03.948462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 73/146] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:03.957517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:03.970362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498079b-2f13-4dd1-81b1-74d8e9e6c574 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:03.978902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:04.045546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.046668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.047339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.049434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.071860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:04.085743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:04.086848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:04.087818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:04.089859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 74/147] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:04.097044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:04.167146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:04.169360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b514a0-ba8c-4e5f-a1ed-ef8082d57fdf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.237868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.238938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.239891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.240389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.241337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:04.241846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.243806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.257689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 74/148] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:04.264134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:04.297037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab5712-4080-489c-972b-724ad491ac7e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.372306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.373316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.374479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.377274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.378236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:04.378735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.381267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.396979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:04.400394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 75/149] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:04.406011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:04.460955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cc5d-eea7-494b-b29f-e38b718bf21d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.531838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:04.534889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccccadb-fa72-4544-a71a-7228d0d15fce None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.559557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.561014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.562013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.562435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] PATH_INFO: `/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.563874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=46dd9c1766844b92b4c8b8eeecf2ee3f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.565480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.571360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 75/150] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:04 2022] GET /identity/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f => generated 289 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:04.571827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.572756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.573037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.573311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.574570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.580257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.581972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 76/151] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:04 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:04.583531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:04.587687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:04.735031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6cc81-a78f-48ce-a6fd-546100b7e55c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.781956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.782652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.782966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.783244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.847347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:04.857788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.860541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.878022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7e811-8a83-4395-a732-883041e86ae8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:04.963511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:04.964403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:04.964689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:04.964969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:04.966225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:04.968252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:04.970865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 76/152] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:04 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:04.976059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:05.001538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:05.001723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 77/153] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:55:05.008119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:05.155674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:05.192952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdb027-3cc7-462e-8ebe-d9336a38dbed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:05.235547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d840b4-b524-4d98-aabe-9fdf5d24f4ca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:05.286074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:05.287367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.287880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.288354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/users/46dd9c1766844b92b4c8b8eeecf2ee3f/roles/96d3a4c0e9534751998d64755241c000` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.290697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, user_id=46dd9c1766844b92b4c8b8eeecf2ee3f, role_id=96d3a4c0e9534751998d64755241c000)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:05.298793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90a77b-9a38-4667-9f81-520d1391190f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:05.302413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:05.303850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.304138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.304416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.307091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:05.308668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7549c80-3a0a-4de0-a1c9-95b1b43d15f2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:05.310450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 78/154] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:05 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:05.312892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449060a-b494-4bc5-99b0-b9f5e0ec61c6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.313186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449060a-b494-4bc5-99b0-b9f5e0ec61c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.313468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449060a-b494-4bc5-99b0-b9f5e0ec61c6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.314708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 79/155] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:05.319981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449060a-b494-4bc5-99b0-b9f5e0ec61c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:05.326743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 77/156] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:04 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/users/46dd9c1766844b92b4c8b8eeecf2ee3f/roles/96d3a4c0e9534751998d64755241c000 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 06:55:05.328878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.329369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.329901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.357419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:05.359523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:05.476281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:05.484274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 78/157] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:05.511805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:05.528902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449060a-b494-4bc5-99b0-b9f5e0ec61c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:05.587891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:05.588909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.589508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.590005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.591271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=9b03423a023143208d210010093cfe64) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:05.591757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:05.593934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:05.597267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:05.604659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 80/158] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:05 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=41985665942d4bba9f8355bf503f068b => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:05.646513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422989-0f1d-43af-bdaf-5105d3568bd1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:05.688636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:05.689509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.689791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.690066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.701656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c37d44b2-828a-4254-8eef-bb7ff1ad4b0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:05.710736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:05.711782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:05.712975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 79/159] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:05.718447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:05.851807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639352a8-4a2c-4048-be1e-74c775711459 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.852404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639352a8-4a2c-4048-be1e-74c775711459 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.852888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639352a8-4a2c-4048-be1e-74c775711459 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.854485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 80/160] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:05.859967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.860559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.861029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.871358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed59178-d9cc-467d-9f6d-1b26961705de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:05.893057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:05.896801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:05.965064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:05.965564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:05.965853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:05.966130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:05.966925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:05.967235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:05.968665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:05.987160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:05.995057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 81/161] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:05.996093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 81/162] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:06.003091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.039629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.117523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c97f0-3d2f-4683-92fd-3d9c8f6ef3c1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:06.170361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:06.172141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.172658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.173192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.174679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:06.176911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7adef-3ce1-41e9-8181-a81b0f860076 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:06.178856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 82/163] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:06.180729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123306-2ab8-4275-b5ed-d1f2b20929bd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.181478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123306-2ab8-4275-b5ed-d1f2b20929bd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.181943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123306-2ab8-4275-b5ed-d1f2b20929bd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.183752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 83/164] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:06.194642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.195301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.196375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.223721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:06.230444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:06.231986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:06.281008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7b2bc-a251-4499-995f-eda0c17a1cb6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:06.311140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:06.330827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 84/165] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:06.332097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.339439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:06.340268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.340559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.340842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.362149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:06.373461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:06.374941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:06.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:06.379492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 82/166] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:06.380388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.543516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f22be4-f434-470b-b59e-9b5e32905cef None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:06.572683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:06.603501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:06.604317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.604607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.604881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e7408-af88-4f74-9dba-20534d855ccd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:06.605005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.621685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:06.631010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:06.631905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:06.632754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:06.635452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 83/167] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:06.636549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.707489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:06.708734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.709167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.709629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.792628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bbd0-c4a6-4feb-aaf9-a4479fb2dfc6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:06.849811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:06.850907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:06.851384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:06.852169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:06.853344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:06.853847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:06.855430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:06.863572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:06.875321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 84/168] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:06.876721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:06.899891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:06.901297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:06.952794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.038574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a899a-7651-4747-a886-8054ae3a81cc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.099274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.101129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.101637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.102398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.103384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:07.103913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.105756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.121472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 85/169] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:07.123531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.202923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 85/170] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 871 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:07.204063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.305781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b2e56-84c2-4d47-97b0-5d07af6b1fcf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.308418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.336763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.351155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49c216-d057-4f53-8730-deb83460f722 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.371647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.372554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.372840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.373116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] PATH_INFO: `/v3/users/9b03423a023143208d210010093cfe64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.375119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] RBAC: Authorizing `identity:get_user(user_id=9b03423a023143208d210010093cfe64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.376628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.378691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 86/171] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/users/9b03423a023143208d210010093cfe64 => generated 275 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:07.379557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.393056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.394040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.394371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.394659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] PATH_INFO: `/v3/users/bb77d87b954348508ac808b62617ddb0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.395939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] RBAC: Authorizing `identity:get_user(user_id=bb77d87b954348508ac808b62617ddb0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.397540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703ece9-7272-49a1-ae2c-68d90d1a4f5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.399312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 86/172] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/users/bb77d87b954348508ac808b62617ddb0 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:07.400518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a148b9-1adf-41ad-864f-429871967535 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.400801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a148b9-1adf-41ad-864f-429871967535 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.401081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a148b9-1adf-41ad-864f-429871967535 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.402688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 87/173] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:07.403598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a148b9-1adf-41ad-864f-429871967535 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.500473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa611a-f695-4de1-8819-b58d30c8a2ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.527933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a148b9-1adf-41ad-864f-429871967535 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.534349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.535233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.535515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.535788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] PATH_INFO: `/v3/projects/d1ccba3669b84275b82a48cb63c5dabe` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.536986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ccba3669b84275b82a48cb63c5dabe)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.538435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07146226-b782-4f72-bc3f-1dac2cc02e77 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.540479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 87/174] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/projects/d1ccba3669b84275b82a48cb63c5dabe => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:07.541715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.541912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.546790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.569198 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:07.571006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:07.584772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.585476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.585755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.586024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.633279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.649002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.686029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.688511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.710699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 88/175] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:07.711476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.804696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 88/176] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:07.806594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:07.841981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59af7fd-c5d5-460d-a015-c606bd646c2c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.868503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.899480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.900128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.900414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.900698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.920046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c35b-f9a3-4288-9a5c-60db4f32f099 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:07.943931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:07.960906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:07.961777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:07.962063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:07.962336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:07.963219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=d1ccba3669b84275b82a48cb63c5dabe, user.id=9b03423a023143208d210010093cfe64) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:07.963522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.965046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.971595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 89/177] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=d1ccba3669b84275b82a48cb63c5dabe&role.id=41985665942d4bba9f8355bf503f068b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:07.974275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:07.975793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:07.979856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.102947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 89/178] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:08.103954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.131827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f759a-b317-445a-a368-c07446d76826 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.156527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.177388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.178627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.178941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.179215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.198499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.200664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9889453a-2dd2-4690-be18-7eb04052f106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:08.210214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:08.212964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 90/179] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:08.213954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20eb03-e023-41e5-855b-34f7a219860e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.214921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20eb03-e023-41e5-855b-34f7a219860e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.214921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20eb03-e023-41e5-855b-34f7a219860e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.219984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 91/180] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:08.219984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7514-96e5-4ca1-8459-a3a8cc06c94a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.219984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7514-96e5-4ca1-8459-a3a8cc06c94a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.219984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d7514-96e5-4ca1-8459-a3a8cc06c94a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.219984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 92/181] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:08.221168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d7514-96e5-4ca1-8459-a3a8cc06c94a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.236414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d763ece-7efa-4624-8e4b-318730194d28 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.274622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.275526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.275810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.276088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.277283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.278740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98769174-d0e8-44cf-93bf-6731cedbaa6d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.281343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 90/182] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.282329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7dc-b5d0-4e25-a41b-9ae3fc4d4832 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.282610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7dc-b5d0-4e25-a41b-9ae3fc4d4832 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.282928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9a7dc-b5d0-4e25-a41b-9ae3fc4d4832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.299683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-24e9a7dc-b5d0-4e25-a41b-9ae3fc4d4832 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:08.300865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-24e9a7dc-b5d0-4e25-a41b-9ae3fc4d4832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:08.343478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d7514-96e5-4ca1-8459-a3a8cc06c94a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.368574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 91/183] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:08.370459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.371226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.371663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.381896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.382882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.383259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.383625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.384895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.386436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.388411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 93/184] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.394213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.397075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:08.399106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:08.446504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.463510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.470264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 92/185] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:08.478961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.504794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9b338-096f-41ce-92e5-8f6bb3411c53 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.540219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.540885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.541163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.541436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.542181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:08.542485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.543895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.552618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 94/186] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.553230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.580170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218a78-ea0f-4483-bdb0-0ee3922af2d0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.613062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.613719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.613999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.614283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.615141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=9b03423a023143208d210010093cfe64) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:08.615457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.616903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.622651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 93/187] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/role_assignments?user.id=9b03423a023143208d210010093cfe64&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=1dfd6d1514104a199766879261c434f7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.623402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.661530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea23175-d4b4-45b6-b809-dabd5533c8ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.672823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.696983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.699236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.700116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.700408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.700738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.701601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=bb77d87b954348508ac808b62617ddb0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:08.701942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.704028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.710823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 95/188] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/role_assignments?user.id=bb77d87b954348508ac808b62617ddb0&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.711692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.756018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d9152-6c65-4569-8cac-e24f4634b3f8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.804874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e615479-ca36-445b-873f-35543d403018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.805081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615479-ca36-445b-873f-35543d403018 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.805883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615479-ca36-445b-873f-35543d403018 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.805883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e615479-ca36-445b-873f-35543d403018 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.822014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e615479-ca36-445b-873f-35543d403018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8e615479-ca36-445b-873f-35543d403018 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:08.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:08.836291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:08.837089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:08.839616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 94/189] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:08.841305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eae20-0fdf-4bba-8048-88927ad050b6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.841830 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eae20-0fdf-4bba-8048-88927ad050b6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.842324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762eae20-0fdf-4bba-8048-88927ad050b6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.843977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 95/190] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:08.844950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eae20-0fdf-4bba-8048-88927ad050b6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.856464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549eae4-31f6-4891-9771-e2acfb2c2805 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.904710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:08.906181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:08.906471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:08.906775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:08.909644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:08.911164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:08.912790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 96/191] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:08.913563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:08.960747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:08.976248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762eae20-0fdf-4bba-8048-88927ad050b6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:08.999751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.011366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.012229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.012513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.012797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.017718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97761ed4-6ffa-457d-bcf9-855e95beaf39 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.027555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.029111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8938c-ba0f-48db-a6c2-70107445a854 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.030847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 96/192] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.032029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.032315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.032601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.048568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:09.049759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:09.051371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.052030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.052309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.052584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.053321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:09.053622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.055020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.063552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 97/193] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.064324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.119289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 97/194] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:09.131271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.176539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de06ae-a970-49f7-8fc7-e3ba8f227129 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.178868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.211225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.211899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.212190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.212471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.213180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.214634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.217126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0a84-c866-422b-8098-8c84191d61b2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.227610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.240477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 98/195] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:09 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:09.241133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.252946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.253876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.254297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.254592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.330385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.331924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.364370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5271861-8b55-4421-ad1a-4ec2c4276bcf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.409373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.410250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.410528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.410851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.412173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.413606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.415500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 99/196] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.416039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.446690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.452149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 98/197] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:09.453024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.468342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.548542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532f99e-e0f5-4224-9cbb-056c4f151c98 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.576211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a65a3f-e067-4eea-8f4e-b5b3813df5b8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.596492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.597294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.597578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.597854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.610353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:09.614020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.614667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.614975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.615253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:09.618118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:09.618990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:09.619799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 100/198] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:09.620448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.705104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.707471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.721562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.726446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.741888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a9db-ed6f-4883-9fa1-538145eb8326 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.779708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.781474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.781755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.782182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.783960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.785435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.787817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 101/199] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.788598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.824682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 99/200] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:09.825759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.914269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214449d5-60d2-4ed4-a99e-e26f7680ad62 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.935135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221f800-01c5-4032-8888-ced2cac9dc21 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:09.947409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.956360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.956965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.957245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.957518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.958264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:09.958566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.959983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.969007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 102/201] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.969796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:09.979433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:09.980124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:09.980406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:09.980682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:09.981516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=2f34ddb7638b41d5aebe218348f21c36) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:09.981821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:09.983478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:09.989606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 100/202] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/role_assignments?user.id=2f34ddb7638b41d5aebe218348f21c36&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:09.990275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:09.991060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:10.081926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e026f29-4282-41ff-ada5-251d2d2bca7a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.107734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b08f8-c269-4601-b54e-c08e705f0a4e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.140171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.141052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.141331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.141604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.142926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.144378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:10.146084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 103/203] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:10.147093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.148045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.148325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.148600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] PATH_INFO: `/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.149940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] RBAC: Authorizing `identity:get_user(user_id=46dd9c1766844b92b4c8b8eeecf2ee3f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.151626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:10.151830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:10.153480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 101/204] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:09 2022] GET /identity/v3/users/46dd9c1766844b92b4c8b8eeecf2ee3f => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:10.159778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:10.244577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:10.261440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628435f-3da4-4eb2-8ad4-8c36d2dc456d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.263140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:10.299230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.300322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.300604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.300876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.301710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=2f34ddb7638b41d5aebe218348f21c36) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:10.302018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.303486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:10.309724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 104/205] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:10 2022] GET /identity/v3/role_assignments?user.id=2f34ddb7638b41d5aebe218348f21c36&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=1dfd6d1514104a199766879261c434f7 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:10.363798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746761f-bee2-4010-837a-425f0d8adee5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.416881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.417917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.418393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.418913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.502102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.503975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:10.557316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:10.621405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 102/206] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:10.627882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:10.759388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bab869-e0cd-4311-98e5-b10ddaad82c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.798884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.799753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.800037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.800319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.801512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.802998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5356082-86ca-4a3e-8110-f8597fabcc54 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:10.805682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 105/207] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:10 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:10.810876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:10.942554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fc3b-8cdc-47a3-8dca-ff37365d4b9c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:10.992590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:10.993244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:10.993529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:10.993808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:10.994763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, scope.project.id=5d1d33a09722400a9530df3eab62138c, user.id=46dd9c1766844b92b4c8b8eeecf2ee3f) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:10.995095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:10.996566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac40d46-2524-44d6-804d-511916a09f86 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:11.007065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 103/208] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:10 2022] GET /identity/v3/role_assignments?user.id=46dd9c1766844b92b4c8b8eeecf2ee3f&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=96d3a4c0e9534751998d64755241c000 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:11.724759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd442b7-ba81-4166-80da-cb6603980ea1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:11.725260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd442b7-ba81-4166-80da-cb6603980ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:11.725368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd442b7-ba81-4166-80da-cb6603980ea1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:11.726335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5d86-5009-46a5-845a-ef75fd78e43e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:11.726703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5d86-5009-46a5-845a-ef75fd78e43e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:11.726822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 106/209] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:11.727189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5d86-5009-46a5-845a-ef75fd78e43e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:11.728569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 104/210] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:11.732633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:11.732932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:11.733250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:11.734378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:11.734997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:11.735495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:11.748196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:11.755091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:11.755669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:11.756952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:11.757453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:11.776066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:11.844957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 105/212] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:11.845231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 107/212] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:11.868214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:11.868821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:11.991630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e4ed8-b155-43f4-9890-4efe34e0f220 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:11.993677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c093f-c6ab-4dcb-b6dd-7691b183a231 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.030062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.031245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.031729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.032213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.034624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.035893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.036235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.036573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.037823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:12.039660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:12.041984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 106/213] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:12.047829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:12.047961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.051568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:12.057041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:12.058729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:12.060047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:12.062783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 108/214] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:12.068871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.081519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.171893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b82295-3698-4d6a-b732-e1527ed0cdcf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.182707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b372-241e-473d-8bfb-c96912fdd9c4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.213215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.213892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.214171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.214448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.215504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:12.217361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:12.225035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.226070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.226673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.227214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.228233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:12.228846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:12.230794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef27ea7-58e2-41b4-8d79-ad7c5e373fcd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:12.246590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 107/215] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:12 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:55:12.246747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 109/216] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:12.255198 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.322002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.372468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ca4d7-819e-40c6-8d75-17dd8da2f2c7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.380308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27443cab-2143-47ac-af64-ef579b7929c1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.382392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27443cab-2143-47ac-af64-ef579b7929c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.389770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27443cab-2143-47ac-af64-ef579b7929c1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.391757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 110/217] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:12 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:12.412932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.412932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.413278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.434003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.434003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.434003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.434003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] PATH_INFO: `/v3/users/e9683e0dee5c42e1a3cee19c79be8e71` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.450102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:12.451362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:12.458377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] RBAC: Authorizing `identity:get_user(user_id=e9683e0dee5c42e1a3cee19c79be8e71)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:12.460324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:12.468151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 108/218] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:12 2022] GET /identity/v3/users/e9683e0dee5c42e1a3cee19c79be8e71 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:12.468458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.559706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.564625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 111/219] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:12.614040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.650837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f8346-cfd5-4712-94a8-f7672f0838dd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.683640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.714079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.714846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.715151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.715433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.781933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c808d9c-d950-40a2-af0e-b5eb4046b196 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:12.812780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:12.814839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f9ccb-7110-422a-a41a-d6c6cdc84058 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:12.826298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:12.828939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.828939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.828939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.844980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a8fb9756-b5e3-432d-9060-8465f24da3a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:12.855045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:12.855962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:12.856739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 112/220] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:12.857881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c55bc-9803-4e61-a1e4-9f2bf5057946 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.858316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c55bc-9803-4e61-a1e4-9f2bf5057946 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.858501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c55bc-9803-4e61-a1e4-9f2bf5057946 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.860419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 113/221] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:12.872171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c55bc-9803-4e61-a1e4-9f2bf5057946 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:12.953727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:12.962452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 109/222] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:12.963659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:12.963957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:12.964237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:12.981714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:12.982969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:12.997516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:13.004810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c55bc-9803-4e61-a1e4-9f2bf5057946 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.044334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.045428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.045984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.046578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.047575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:13.048075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.049859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.062883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 110/223] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:13.062883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.068681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 114/224] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:13.074482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.164676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8db6d3-4133-48d7-b2a6-b56741442dcc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.198243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811c9d4-b1a6-4c71-a3c7-a61ea9805aec None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.203546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.204425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.204708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.204985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.206183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.208094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.212017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 111/225] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:13.213006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.231123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:13.232429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:13.246113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.247026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.247412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.247699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.249565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.251061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.258201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 115/226] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:13.258201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.351985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfeb8a9-2b7a-4339-8807-adce962daeb7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.391413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.392574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.393057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.393547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.395043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc418f-b84a-4054-8c34-09e40178a446 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.405044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:13.413387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:13.414285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:13.415099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:13.417547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 112/227] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:13.418527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.437654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.438326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.438614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.438936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.440050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=e9683e0dee5c42e1a3cee19c79be8e71) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:13.440368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.442488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.450124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 116/228] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/role_assignments?user.id=e9683e0dee5c42e1a3cee19c79be8e71&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=41985665942d4bba9f8355bf503f068b => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:13.455045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.557638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:13.570358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:13.608963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d950b4e-dd0e-482e-b606-3e7c4be274a7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.659303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139c846-d9de-4a90-bee8-c3e2472c67a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.678868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.680104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.680590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.681064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.682805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.684463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.687089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 113/229] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:13.693314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.709670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.710342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.710618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.710937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.711801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:13.712109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.713496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.734831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 117/230] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:13.739807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:13.814751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ffdae-45c2-4c6c-abeb-3c190e4c021c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.855322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:13.856961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:13.857450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:13.858045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/2f34ddb7638b41d5aebe218348f21c36/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:13.861296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, user_id=2f34ddb7638b41d5aebe218348f21c36, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:13.863012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:13.882147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 114/231] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:13 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/2f34ddb7638b41d5aebe218348f21c36/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:13.966642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68198777-23e7-484e-8e59-acf65a95db2a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:13.997289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:14.020596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:14.021469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.021757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.022035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.027533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:14.029001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:14.031270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 118/232] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:13 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:14.037179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:14.041549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:14.167087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c02c-1f4c-4c7e-a19c-9b8de3c8345d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:14.206170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:14.207338 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.207832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.208256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.210543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:14.212197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143444e7-16dd-4288-a8e3-302dd07013b4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:14.215028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 115/233] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:14 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:14.221015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:14.330166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08539a60-a8cf-4c23-882a-997d2631d3f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:14.366160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:14.367603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.368033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.368455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/2f34ddb7638b41d5aebe218348f21c36/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, user_id=2f34ddb7638b41d5aebe218348f21c36, role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:14.372514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d0899-a1d4-41e0-9513-f312af2507fe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:14.406646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 119/234] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:14 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/2f34ddb7638b41d5aebe218348f21c36/roles/1dfd6d1514104a199766879261c434f7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 06:55:14.586763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bc27-07b8-446f-80c3-b5f3ebd0aeb7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.587285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bc27-07b8-446f-80c3-b5f3ebd0aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.587735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6bc27-07b8-446f-80c3-b5f3ebd0aeb7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.589520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 116/235] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:14.595392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7b784-68f7-49e2-83db-21daa72acf5f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.595871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7b784-68f7-49e2-83db-21daa72acf5f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.596298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7b784-68f7-49e2-83db-21daa72acf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.619536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-55d7b784-68f7-49e2-83db-21daa72acf5f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:14.620992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-55d7b784-68f7-49e2-83db-21daa72acf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:14.665603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:14.705093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 120/236] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:14.730238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6bc27-07b8-446f-80c3-b5f3ebd0aeb7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:14.819894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:14.854371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6bc27-07b8-446f-80c3-b5f3ebd0aeb7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:14.899560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:14.900704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.901133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.901576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.907503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92fc6-d2f7-409a-9b16-98db8d1045b9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.907792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92fc6-d2f7-409a-9b16-98db8d1045b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.908071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92fc6-d2f7-409a-9b16-98db8d1045b9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.909466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 121/237] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:14.914548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:14.914862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:14.915147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:14.918443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:14.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:14.927995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:14.929060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:14.931180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 117/238] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:14.936912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:14.937950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:14.938533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:15.061054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97216d99-c562-49c2-a93c-7058593ef785 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.095014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:15.100264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 122/239] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:15.109071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.109687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.109974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.110252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.111040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:15.111349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.112747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.126304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:15.127524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 118/240] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:15.129105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:15.279879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b9463-ae76-4c5b-9f37-0007b915d7a9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.290813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8860c2-226b-4b14-b31f-e37924606624 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.299118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:15.340990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.341886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.342173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.342465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.344141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.345590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.347528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 123/241] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:15.352776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:15.363522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.364350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.364635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.364915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] PATH_INFO: `/v3/users/047ce0e662054c81a4d8e6a857b8b09b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.391533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] RBAC: Authorizing `identity:get_user(user_id=047ce0e662054c81a4d8e6a857b8b09b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.393029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.394812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 119/242] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:15 2022] GET /identity/v3/users/047ce0e662054c81a4d8e6a857b8b09b => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:15.400309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:15.456478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8dc60-e067-4dd1-ada4-8a218109e7a0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.480567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:15.497932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.499554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.500054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.500525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.501430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.503198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.525359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 124/243] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:15 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:15.557852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82495c3-4d78-40ed-b4e2-efa1c1c15680 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.602469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.603174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.603459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.603740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.692301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.694133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.762292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:15.816969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 120/244] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:55:15.822614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:15.855336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:15.925660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc86f7-7440-41fa-ac97-dd2e4956a9f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:15.963443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:15.964225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:15.964503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:15.964783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:15.965973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:15.967451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b945d16-f3e1-4395-917b-80344ea07a68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:15.970469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 125/245] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:15 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:15.975940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.055254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dee6a-b749-4d80-968a-371bebf7e8b2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.055543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dee6a-b749-4d80-968a-371bebf7e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.055821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186dee6a-b749-4d80-968a-371bebf7e8b2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.058745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 126/246] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:16.063352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.063641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.063921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.082578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:16.085904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:16.102005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e6859-fd35-47d2-be11-6511377122f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:16.137693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:16.138949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.138949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.139235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.140001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=047ce0e662054c81a4d8e6a857b8b09b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:16.140309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:16.141888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:16.149119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 121/247] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:15 2022] GET /identity/v3/role_assignments?user.id=047ce0e662054c81a4d8e6a857b8b09b&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=96d3a4c0e9534751998d64755241c000 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:16.176348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 127/248] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:16.200833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.222310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:16.320862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d12da-26b7-4438-a336-b687924fbcdd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:16.367615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:16.368430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.368711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.368989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.384932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:16.392932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:16.393803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:16.395054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 122/249] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:16.400068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.437794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:16.529065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908fb42-14bc-4d62-b708-c151d0b93b87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:16.564433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:16.565114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.565402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.565684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.566453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:16.566784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:16.568184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2eccce-2c65-4a59-bbbe-2f2d00b2200b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:16.577727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 128/250] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:16.583243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.680018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:16.704459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88eaf8-d7bd-49fb-b137-1a697efe37fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:16.719542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60268-04ab-42c5-a6e4-b6e4eec7b1b5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.719839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60268-04ab-42c5-a6e4-b6e4eec7b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.720103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf60268-04ab-42c5-a6e4-b6e4eec7b1b5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.721714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 129/251] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:55:16.727910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.728199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.728485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.754828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:16.757304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:16.779942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:16.781327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:16.781611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:16.781888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] PATH_INFO: `/v3/users/e9683e0dee5c42e1a3cee19c79be8e71` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:16.787263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] RBAC: Authorizing `identity:get_user(user_id=e9683e0dee5c42e1a3cee19c79be8e71)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:16.788727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:16.790616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 123/252] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:16 2022] GET /identity/v3/users/e9683e0dee5c42e1a3cee19c79be8e71 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:16.808334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.890312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 130/253] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:16.917417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:16.950592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9259-b98f-4472-aca3-49ff93af3921 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:16.957430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:16.997809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:16.999932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.000462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.000939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.007510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:17.009185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:17.011169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 124/254] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:16 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:17.017621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.043568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:17.117564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbb50c-b8b5-4534-8ef7-b510dc924b38 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:17.184407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:17.185213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.185498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.185779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.198318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f90ea-7ca0-42d6-b3d1-fdf2614c2410 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:17.219378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:17.227059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:17.227942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:17.229148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 131/255] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:17.234665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.261104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:17.262002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.262265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.262539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/e9683e0dee5c42e1a3cee19c79be8e71/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.264615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, user_id=e9683e0dee5c42e1a3cee19c79be8e71, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:17.266056 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae771d-d574-4bc1-90ca-c76d4d095faa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:17.281226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 125/256] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:17 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/e9683e0dee5c42e1a3cee19c79be8e71/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:17.282717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06e23d-4c26-4a29-a5cb-9ed88b9e3925 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.283033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06e23d-4c26-4a29-a5cb-9ed88b9e3925 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.283315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06e23d-4c26-4a29-a5cb-9ed88b9e3925 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.285168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 126/257] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:17.290989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.291323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.291598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.313749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:17.315277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:17.397646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 127/258] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:17.421990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:17.442188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.453175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dd3d7-2ea5-4ad3-93a5-2808ad2fee48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:17.475585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:17.525353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:17.526388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.526931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.527542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.528542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:17.528860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:17.530410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:17.543501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 132/259] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:17.551615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.604579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71876f4a-61e9-4e6d-8497-9a9d5c87bf72 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:17.649753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:17.651115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.651568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.651989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.666638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:17.676567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:17.677486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:17.678265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:17.681077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 128/260] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:17.690320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.772852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17549fd1-0f2b-4f27-aced-4a90d26dcfb1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:17.838044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:17.843567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:17.844586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:17.844868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:17.845139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:17.847617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:17.849074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:17.851035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 133/261] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:17 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:17.856329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:17.871769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:17.946419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af60c3-17d3-4d47-9733-0af0792905d7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:18.019631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:18.021254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:18.021750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:18.022224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:18.023208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:18.024063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:18.025692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:18.041506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 129/262] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:18.047764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:18.068949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634093b-d591-4f3f-b9b4-c315573bdab2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:18.141874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:18.142678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:18.144746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:18.145025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:18.259600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9c3b2-d27c-451e-8669-5aac7adc8872 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:18.308353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:18.309828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:18.312537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:18.331108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:18.332337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:18.333214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:18.333491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:18.333766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] PATH_INFO: `/v3/users/2f34ddb7638b41d5aebe218348f21c36` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:18.335875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f34ddb7638b41d5aebe218348f21c36)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:18.337314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:18.340275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 130/263] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:18 2022] GET /identity/v3/users/2f34ddb7638b41d5aebe218348f21c36 => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:18.346778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:18.476669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe0d76-bbee-49a4-a60c-b6a5b9a5e3a2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:18.516864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 134/264] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:18.521324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:18.521324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:18.521324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:18.521324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:18.524571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:18.637683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:18.663589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:18.665376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:18.747973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bab11-e4f3-4311-b997-1f3a4972bbfc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:18.772741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:18.822057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 131/265] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:18.827927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:18.831616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:18.832459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:18.832744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:18.833020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:18.834255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:18.835743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:18.846802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 135/266] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:18 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:18.855836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:18.990731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.012877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d443c94-1d02-40d1-a5cd-cd44d40be372 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.025037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c886-842a-4b1c-a3f9-16379fbb17ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.077481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.078360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.078645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.078998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.080218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:19.081688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:19.083557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 132/267] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:18 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:19.089479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.092007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.092708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.092995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.093380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.094320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=2f34ddb7638b41d5aebe218348f21c36) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:19.094640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:19.096090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17630da5-ea69-4776-b927-6f3c72daed43 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:19.106978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 136/268] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:18 2022] GET /identity/v3/role_assignments?user.id=2f34ddb7638b41d5aebe218348f21c36&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:19.168121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab932-f740-4889-919a-5bbe51dfa969 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.168512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab932-f740-4889-919a-5bbe51dfa969 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.168797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab932-f740-4889-919a-5bbe51dfa969 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.170587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 137/269] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:55:19.192699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.192994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.193253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.239127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f9521-19b8-4dd4-b1b6-b52347a72299 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.248023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:19.249299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:19.276854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.277450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.277738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.278020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.278901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=2f34ddb7638b41d5aebe218348f21c36) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:19.279279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:19.281330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f8240-6eb4-4be1-bd73-79658b57c9df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:19.287955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 133/270] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/role_assignments?user.id=2f34ddb7638b41d5aebe218348f21c36&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=1dfd6d1514104a199766879261c434f7 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:19.334579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292823c9-bcdb-4bca-9d6f-1761d5471136 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.334905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292823c9-bcdb-4bca-9d6f-1761d5471136 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.335180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292823c9-bcdb-4bca-9d6f-1761d5471136 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.337526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 134/271] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:55:19.343727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.344028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.344271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.355158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.374931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:19.376145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:19.424026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 138/272] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:19.425754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.459862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.481613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 135/273] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:19.529457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.611512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9d9dc-5ab2-4edf-a91e-f86114373ea4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.653168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.653971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.654514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.654514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.660931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a496b06-7174-4b05-9d82-406b4e228a0b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.666377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c6f36a7c-d91b-4427-af64-76408239bbe8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:19.674611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:19.675520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:19.676330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 139/274] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:19.682624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836678e-4ca1-4598-8d2b-60c3b826c645 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.683111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836678e-4ca1-4598-8d2b-60c3b826c645 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.683563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836678e-4ca1-4598-8d2b-60c3b826c645 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.685582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 140/275] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:19.686492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836678e-4ca1-4598-8d2b-60c3b826c645 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.701025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.701845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.702673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.703358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.713797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ea086612-e5e5-4011-95e4-a19ec14a1499 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:19.722114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:19.723128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 136/276] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:19.724666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.725422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.725422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.728693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.742290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:19.743902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:19.753839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.810023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836678e-4ca1-4598-8d2b-60c3b826c645 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.823149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 137/277] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:19.823149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.852714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.853378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.853659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.853934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.854677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:19.855012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:19.856389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:19.866888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 141/278] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:19.866888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:19.940831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04142d-74f5-4981-8108-c068dd2d8270 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.970748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:19.977784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:19.978603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:19.978918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:19.979195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:19.979942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:19.980247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:19.981073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e27c9-eea8-46d9-99f7-1566ef2bef23 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:19.981630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:19.990676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 138/279] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:19.991333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.001340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.025616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.026327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.026616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.026922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.028056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.029543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.031578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 142/280] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.032178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.099908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6aaac2-aeba-492a-97c4-cc6746d2a2ff None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.133044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.133340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632add0e-22cc-4e64-994e-49eaf3be10cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.133900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.134187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.134464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] PATH_INFO: `/v3/users/047ce0e662054c81a4d8e6a857b8b09b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.135691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] RBAC: Authorizing `identity:get_user(user_id=047ce0e662054c81a4d8e6a857b8b09b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.137142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.138783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 139/281] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:19 2022] GET /identity/v3/users/047ce0e662054c81a4d8e6a857b8b09b => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.141870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.168324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.169196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.169458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.169730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] PATH_INFO: `/v3/users/670a3c3f3dd849dbbb1d052adcbae65b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.183970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] RBAC: Authorizing `identity:get_user(user_id=670a3c3f3dd849dbbb1d052adcbae65b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.185475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2883782-be9f-43a2-b6e0-4ccd6c86a19c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 143/282] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/users/670a3c3f3dd849dbbb1d052adcbae65b => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1600366-e7f6-45e4-8443-78c5d0e618d0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1600366-e7f6-45e4-8443-78c5d0e618d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1600366-e7f6-45e4-8443-78c5d0e618d0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 144/283] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:20.187747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1600366-e7f6-45e4-8443-78c5d0e618d0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.211362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.227447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.252256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776f9a-6a41-4d40-a87c-9626d2390564 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.284185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.284847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.285128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.285401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.286102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.287549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.292931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1600366-e7f6-45e4-8443-78c5d0e618d0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.306398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 140/284] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:20 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:20.307496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.340906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.341787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.342071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.342355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.343654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.345199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22968b8f-9bbb-4417-b995-f3cc24124483 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.346989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 145/285] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.348308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.348618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.348875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.366551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:20.367809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:20.426867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedd087-15d6-413a-ad30-03a1337819d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.442368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 146/286] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:20.443546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.466607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.471838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.480350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.481008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.481288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.481563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.482398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, user.id=670a3c3f3dd849dbbb1d052adcbae65b, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:20.482707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.484764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.490023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 141/287] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/role_assignments?user.id=670a3c3f3dd849dbbb1d052adcbae65b&scope.system=all&role.id=41985665942d4bba9f8355bf503f068b => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.490939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.570633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a72159-9cd1-4f5e-9e5f-ddcbce753edb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.605103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c59be-80f0-4dcc-855c-463d53f8433f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.610901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.612161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.612656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.613045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/047ce0e662054c81a4d8e6a857b8b09b/roles/96d3a4c0e9534751998d64755241c000` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.615158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, user_id=047ce0e662054c81a4d8e6a857b8b09b, role_id=96d3a4c0e9534751998d64755241c000)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.616805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.634178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 147/288] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:20 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/users/047ce0e662054c81a4d8e6a857b8b09b/roles/96d3a4c0e9534751998d64755241c000 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:20.647496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.648488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.648765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.649043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.663131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:20.672115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:20.673114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:20.673908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 142/289] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:20.682795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.768074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.794419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97745-aff1-4dd9-be12-cfe8dfa9b42a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.834181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.834878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.835164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.835436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.836187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:20.836546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:20.838045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:20.848030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 148/290] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:20.853537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:20.913021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:20.960403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd59bed-c8b4-4e6c-8645-2d4aae4daefa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:20.996024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:20.997049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:20.997393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:20.997764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] PATH_INFO: `/v3/users/e9683e0dee5c42e1a3cee19c79be8e71` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:20.999094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e9683e0dee5c42e1a3cee19c79be8e71)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:21.000603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:21.002658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 143/291] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:20 2022] GET /identity/v3/users/e9683e0dee5c42e1a3cee19c79be8e71 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:21.007990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:21.149601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59db76-e4fa-4d34-9e73-22c59ffe7ffb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:21.205272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:21.205855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:21.206144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:21.206412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:21.221138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:21.298421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:21.299889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:21.402318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 149/292] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:21.408671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:21.534109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137553e4-1945-4730-81ef-e87279ac62c3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:21.557177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:21.570360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:21.571590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:21.572076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:21.572558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:21.573976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:21.575679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e478f81c-d0a1-4849-b20a-5be851eb58dd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:21.578246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 144/293] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:21 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:21.584379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:21.627829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:21.697460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464dcfe-3d2e-4db6-99f5-85fb07f793cf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:21.744284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:21.745269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:21.745751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:21.746248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:21.747300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=e9683e0dee5c42e1a3cee19c79be8e71) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:21.747813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:21.749997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90102a97-96d6-4edf-ae32-39484c83e4e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:21.759431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 150/294] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:21 2022] GET /identity/v3/role_assignments?user.id=e9683e0dee5c42e1a3cee19c79be8e71&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=41985665942d4bba9f8355bf503f068b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:22.782946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bac4-9984-443b-a02f-ed8aaed39455 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:22.783907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bac4-9984-443b-a02f-ed8aaed39455 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:22.784409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bac4-9984-443b-a02f-ed8aaed39455 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:22.786237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 145/295] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:22.791185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdcc26-864d-42ee-a200-49bb66c4a49e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:22.791576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdcc26-864d-42ee-a200-49bb66c4a49e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:22.791736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdcc26-864d-42ee-a200-49bb66c4a49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:22.809326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-64cdcc26-864d-42ee-a200-49bb66c4a49e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:22.810504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64cdcc26-864d-42ee-a200-49bb66c4a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:22.881908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 151/296] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:22.904646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bac4-9984-443b-a02f-ed8aaed39455 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.015184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bac4-9984-443b-a02f-ed8aaed39455 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.017312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bf34-3c4b-46e6-b6c1-fbeeb9e42d1f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.017603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bf34-3c4b-46e6-b6c1-fbeeb9e42d1f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.017882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0bf34-3c4b-46e6-b6c1-fbeeb9e42d1f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.019507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 152/297] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:23.024522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc9b0-1a0d-4f33-963f-9c2c2357cbfe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.024806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc9b0-1a0d-4f33-963f-9c2c2357cbfe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.025086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc9b0-1a0d-4f33-963f-9c2c2357cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.042188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2a0fc9b0-1a0d-4f33-963f-9c2c2357cbfe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:23.043379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a0fc9b0-1a0d-4f33-963f-9c2c2357cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:23.049989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38988159-5892-453c-a443-ef99df472f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.050912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988159-5892-453c-a443-ef99df472f81 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.051232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988159-5892-453c-a443-ef99df472f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.051513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988159-5892-453c-a443-ef99df472f81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.062573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38988159-5892-453c-a443-ef99df472f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-38988159-5892-453c-a443-ef99df472f81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:23.070234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:23.071327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 146/298] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:23.076072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000718-c299-444c-ae87-432277fe0466 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.076784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000718-c299-444c-ae87-432277fe0466 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.076784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000718-c299-444c-ae87-432277fe0466 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.076784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 147/299] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:23.077597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.078965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5000718-c299-444c-ae87-432277fe0466 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.121799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 153/300] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:23.123192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49331-9aa8-4748-9997-61bf2963a641 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.123708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49331-9aa8-4748-9997-61bf2963a641 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.124747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49331-9aa8-4748-9997-61bf2963a641 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.127024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 154/301] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:23.127931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b6828-1d1b-48de-a1fd-d7b9b7df71bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.128189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b6828-1d1b-48de-a1fd-d7b9b7df71bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.128471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b6828-1d1b-48de-a1fd-d7b9b7df71bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.136303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.145431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-224b6828-1d1b-48de-a1fd-d7b9b7df71bf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:23.146689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-224b6828-1d1b-48de-a1fd-d7b9b7df71bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:23.192882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5000718-c299-444c-ae87-432277fe0466 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.216504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 155/302] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:23.217807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.218093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.218370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.231400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.232065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.232349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.232625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.233913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:23.234185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:23.234281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.235362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:23.235616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.245109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 148/303] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:23.245904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.301945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 156/304] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:23.302703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.333531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.344300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550611c-df9a-43a4-9f91-6718380072d9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.356949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.377334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.378116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.378393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.378664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.388285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:23.396436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:23.397344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:23.398124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 149/305] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:23.398885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.412582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d1c30-1804-4506-8bcb-85d853730362 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.446374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.447183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.447459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.447728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.457920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:23.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:23.466337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 157/306] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:23.467463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.497704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40881538-8a9d-436f-b064-be7cef4e7377 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.529931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.531377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.531653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.531924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] PATH_INFO: `/v3/users/670a3c3f3dd849dbbb1d052adcbae65b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.533092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=670a3c3f3dd849dbbb1d052adcbae65b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.534531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.536172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 150/307] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/users/670a3c3f3dd849dbbb1d052adcbae65b => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:23.536921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.550967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.569132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46e482-8b23-4f27-9d01-5acf67ca730a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.579303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.603449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.604223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.604779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.605063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.606212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.607689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.609325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 158/308] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:23.610077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.637076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0611927-7c2b-4397-9391-d5f30b948ac3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.669983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.670679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.670991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.671271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.672020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:23.672332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.673694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.681528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 151/309] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:23.687492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.708023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccee0b-82c7-463e-bbde-483daa099ce3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.740230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.740881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.741162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.741438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.742178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:23.742483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.743881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.752129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 159/310] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:23.752874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.768600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.784641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc916-6fe7-4c56-ac03-d12ae9c5f207 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.786133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:23.819516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.820296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.820574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.820846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] PATH_INFO: `/v3/system/users/670a3c3f3dd849dbbb1d052adcbae65b/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.822309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=670a3c3f3dd849dbbb1d052adcbae65b, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.823696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.832372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 152/311] 10.222.0.48 () {58 vars in 1334 bytes} [Fri Jul 1 06:55:23 2022] PUT /identity/v3/system/users/670a3c3f3dd849dbbb1d052adcbae65b/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:23.833172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.857427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c85-a8f3-4481-b400-a000d9fc6678 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:23.897821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:23.898507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:23.898872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:23.899081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] PATH_INFO: `/v3/groups` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:23.899901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:23.901495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:23.917763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 160/312] 10.222.0.48 () {60 vars in 1134 bytes} [Fri Jul 1 06:55:23 2022] POST /identity/v3/groups => generated 229 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:23.919054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:23.968775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04f672-3a02-46f9-a708-0b756437c120 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.019738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.021287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.021287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.021287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] PATH_INFO: `/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.028418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:24.041841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] RBAC: Authorizing `identity:get_user(user_id=2ccb7f1d1f2c4101ac99c0e06cf6cba3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.043128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.044968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 153/313] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:24.051465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:24.076225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dbe90-87dc-4f3f-870c-abeb3d7d9c64 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.079808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:24.124650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.125556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.125843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.126120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] PATH_INFO: `/v3/users/047ce0e662054c81a4d8e6a857b8b09b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.127349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] RBAC: Authorizing `identity:get_user(user_id=047ce0e662054c81a4d8e6a857b8b09b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.129487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.130432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 161/314] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:23 2022] GET /identity/v3/users/047ce0e662054c81a4d8e6a857b8b09b => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:24.135784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:24.183819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c4535-bac6-401a-956e-3de32610c674 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.229329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.230103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.230447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.230811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.232169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, user.id=2ccb7f1d1f2c4101ac99c0e06cf6cba3, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:24.233033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.235374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.242456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 154/315] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Jul 1 06:55:24 2022] GET /identity/v3/role_assignments?user.id=2ccb7f1d1f2c4101ac99c0e06cf6cba3&scope.system=all&role.id=96d3a4c0e9534751998d64755241c000 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:24.262402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5707c0-af13-4a26-968f-8dfa56bab7fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.302399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.303103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.303384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.303658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.317731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:24.382666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.384218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.489748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 162/316] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:24.496067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:24.586536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:24.600775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0404-95a6-49c4-8fc1-441873353fb4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.639514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.640297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.640577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.640852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.642043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.643513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d380d-6328-4be4-8f09-fa2dc25a765b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.645583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 155/317] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:24 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:24.651955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:24.696142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:24.756376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dd8fc-e3a8-461b-8ff3-4a278cfdc9c4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:24.791507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:24.792163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:24.792440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:24.792711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:24.793528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, scope.project.id=fa7b086c98dd452db4945dd06282fd24, user.id=047ce0e662054c81a4d8e6a857b8b09b) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:24.793831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:24.795345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee913c-0e8c-4708-9158-b4bb26a6f432 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:24.801549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 163/318] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:24 2022] GET /identity/v3/role_assignments?user.id=047ce0e662054c81a4d8e6a857b8b09b&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=96d3a4c0e9534751998d64755241c000 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:25.949964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6ed52-dfcc-4e66-ba7e-f36545f3a3c7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:25.950493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6ed52-dfcc-4e66-ba7e-f36545f3a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:25.950493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6ed52-dfcc-4e66-ba7e-f36545f3a3c7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:25.952634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 156/319] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:25.957628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b0247-2dee-4f07-8a94-114fed9e23b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:25.958048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b0247-2dee-4f07-8a94-114fed9e23b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:25.958207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b0247-2dee-4f07-8a94-114fed9e23b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:25.974646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e3b0247-2dee-4f07-8a94-114fed9e23b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:25.975845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e3b0247-2dee-4f07-8a94-114fed9e23b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:26.043084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 164/320] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:26.048675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947041c-4c03-4552-82a9-484c3dd5f2bf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.048967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947041c-4c03-4552-82a9-484c3dd5f2bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.049249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947041c-4c03-4552-82a9-484c3dd5f2bf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.050608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 157/321] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:26.055877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.056173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.056455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.066500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947041c-4c03-4552-82a9-484c3dd5f2bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.074454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:26.076101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:26.105568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.148397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 165/322] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:26.170795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.174948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947041c-4c03-4552-82a9-484c3dd5f2bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.213194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.213982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.214261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.214531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.224915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:26.232464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:26.233902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:26.234700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 158/323] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:26.240154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.258788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.291437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe20ed-e450-4f3d-a90c-c97b36db0b39 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.327441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.328228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.328511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.328788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.329960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.331528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2775f50-5253-45f7-92b4-76dc2f685b53 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.333147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 166/324] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:26.334303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727254-0e2d-4b82-8e52-81588408945e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.334592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727254-0e2d-4b82-8e52-81588408945e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.334902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727254-0e2d-4b82-8e52-81588408945e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.336137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 167/325] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:26.338716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b727254-0e2d-4b82-8e52-81588408945e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.341928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb2359-355e-479f-ad9f-1441bec6fb29 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.369217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.375509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.376901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.377186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.377461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.378212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:26.378522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.379927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d98a9-7882-4939-8c31-566fd818258c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.392522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 159/326] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:26.392522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.392522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.392522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.405710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:26.406923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:26.440794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b727254-0e2d-4b82-8e52-81588408945e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.472283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 160/327] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:26.472892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.481606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.482265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.482387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.482546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.483764 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] PATH_INFO: `/v3/groups` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.484463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.485915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.502800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 168/328] 10.222.0.48 () {60 vars in 1134 bytes} [Fri Jul 1 06:55:26 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:55:26.502800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.595112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0b919-85ce-4c16-bb00-48f36bb6d25f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.603279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.615032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197d803-dfc4-4951-aba7-9d81a383ec31 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.633785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.634674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.634980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.635257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] PATH_INFO: `/v3/users/670a3c3f3dd849dbbb1d052adcbae65b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.636507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] RBAC: Authorizing `identity:get_user(user_id=670a3c3f3dd849dbbb1d052adcbae65b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.637976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.639885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 161/329] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/users/670a3c3f3dd849dbbb1d052adcbae65b => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:26.645478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.653667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.654458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.654729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.655600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.669628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:26.678100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:26.679029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:26.679843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 169/330] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:26.685518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:26.738262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:26.766984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fc229-3e2f-433d-a1dd-4595e7c60d7c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.805786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f388f0d-1b21-48fd-ade1-0db73fcaa69f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:26.815425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.816041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.816316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.816587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.817419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] RBAC: Adding query filter params (role.id=41985665942d4bba9f8355bf503f068b, user.id=670a3c3f3dd849dbbb1d052adcbae65b, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:26.817727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.819203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.836944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 162/331] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/role_assignments?user.id=670a3c3f3dd849dbbb1d052adcbae65b&scope.system=all&role.id=41985665942d4bba9f8355bf503f068b => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:26.847616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:26.848299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:26.848585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:26.848890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:26.849633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:26.849947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:26.852008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:26.862759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 170/332] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:26.869095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:27.001085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82755d8-8d21-4c8b-ad61-5aef0ab5d8e0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:27.039835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:27.040723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:27.041009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:27.041287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] PATH_INFO: `/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:27.042500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] RBAC: Authorizing `identity:get_user(user_id=2ccb7f1d1f2c4101ac99c0e06cf6cba3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:27.043994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547f0ac-d307-42d3-ad4a-6570a6f5ae73 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:27.045792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 163/333] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:26 2022] GET /identity/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:27.051354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:27.069034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:27.169688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fda9ef-a466-49a3-b9a8-6b274294f257 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:27.215858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:27.217005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:27.217306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:27.218190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] PATH_INFO: `/v3/system/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3/roles/96d3a4c0e9534751998d64755241c000` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:27.219716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ccb7f1d1f2c4101ac99c0e06cf6cba3, role_id=96d3a4c0e9534751998d64755241c000)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:27.221086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd82ff-a2f5-4ab6-a813-0516529463d9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:27.233494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 171/334] 10.222.0.48 () {58 vars in 1334 bytes} [Fri Jul 1 06:55:27 2022] PUT /identity/v3/system/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3/roles/96d3a4c0e9534751998d64755241c000 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:28.779001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412863db-beb3-4d27-b11f-8bc0e4a5038a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:28.780719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412863db-beb3-4d27-b11f-8bc0e4a5038a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:28.781205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412863db-beb3-4d27-b11f-8bc0e4a5038a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:28.783299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 164/335] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:28.792418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:28.793299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:28.793777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:28.813299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:28.814852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:28.901434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 172/336] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:28.943205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412863db-beb3-4d27-b11f-8bc0e4a5038a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:28.946543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:29.063513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412863db-beb3-4d27-b11f-8bc0e4a5038a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:29.101356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:29.102577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.103084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.103550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.119632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:29.127915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:29.128811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:29.130030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 165/337] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:29.135199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:29.151722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:29.286166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32241-ee2b-4911-8eff-dbc527cfa0ab None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:29.332611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:29.333595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.334121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.334586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.335588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:29.336126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:29.337672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9ee6-0e55-4066-8193-cdaf072e9ee7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:29.350514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 173/338] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:29.355977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:29.448985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:29.485023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de510193-abf2-4be3-ae80-1e94232bb8b8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.485368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de510193-abf2-4be3-ae80-1e94232bb8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.485592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de510193-abf2-4be3-ae80-1e94232bb8b8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.487231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 174/339] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:29.492424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770a34-8c39-4933-8cd2-423c31ad8a81 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.492958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770a34-8c39-4933-8cd2-423c31ad8a81 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.493507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770a34-8c39-4933-8cd2-423c31ad8a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.497521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573140b-afd5-4b26-b586-7d106a0c9045 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:29.514932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70770a34-8c39-4933-8cd2-423c31ad8a81 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:29.518622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70770a34-8c39-4933-8cd2-423c31ad8a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:29.559816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:29.560880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.561941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.562418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.642971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:29.645168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:29.683150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:29.690460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 175/340] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:29.695262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f371c-794a-40c2-99fb-2766ee5d94d2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.695737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f371c-794a-40c2-99fb-2766ee5d94d2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.696029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f371c-794a-40c2-99fb-2766ee5d94d2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.697390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 176/341] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:29.703676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.703962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.704245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.724570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:29.730548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:29.733812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:29.773834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 166/342] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:29.775065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:29.904635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c0f46-e5d1-46c8-82df-5a6cbf529d2f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:29.934915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 177/343] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:29.935975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:29.955532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:29.956590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:29.957019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:29.957441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:29.980848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:29.989808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:29.990904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:29.991921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:30.003620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 167/344] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:30.007529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.060453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.140236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1346e4-a018-4b25-ba3a-e966a58953ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.182164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.193157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.193364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f091d2e-2ecd-4a33-9eb5-b005ff0e7070 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.194459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.195035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.195750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.198431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.200136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.202606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 178/345] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:29 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:30.204372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.240488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.241272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.241713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.242152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.255207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-adeb0b62-8525-418f-8897-039d87461312 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:30.263148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:30.264245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:30.265887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:30.268496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 168/346] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:30.270072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463819a1-71bb-4332-be6e-3d780f2c98c5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.270494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463819a1-71bb-4332-be6e-3d780f2c98c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.270947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463819a1-71bb-4332-be6e-3d780f2c98c5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.272352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 169/347] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:30.273208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463819a1-71bb-4332-be6e-3d780f2c98c5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.328626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4192-0891-4f09-8ae4-3fc248ade03e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.363219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.364138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.364591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.365010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.365585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.366074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:30.366529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.368095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b450c-eb37-4ea9-9629-1b540e7135d6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.378642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 179/348] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:30.380254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.380747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.381229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.388355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463819a1-71bb-4332-be6e-3d780f2c98c5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.401192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:30.402522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:30.428773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.429913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.430353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.430787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.438228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.439871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.442110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 170/349] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:30.443120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.471167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.480704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 180/350] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:30.482017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.554607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0936f-3534-4f74-895d-6c962a771276 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.593031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.593629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a1c4-b2ef-4f37-a87c-640d8f83295c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.593741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.594014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.594291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.595072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:30.595376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.597723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.608108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 171/351] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:30.609164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.623189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.636954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.637996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.638485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.639024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.712258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.724444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.726227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.740486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6983c-11ed-4b9e-84e1-2d9351548c8b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.786330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.787353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.787862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.788872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.790200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=5d1d33a09722400a9530df3eab62138c) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:30.790798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:30.793421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:30.801482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 172/352] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=41985665942d4bba9f8355bf503f068b => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:30.802708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.845044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 181/353] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:30.846146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:30.890179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.923857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978c07a-05ca-4193-b540-fc60983eb25a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.956734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:30.964851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:30.965725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.966073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.966538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.976682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213f69d-3d23-4114-88ab-b0eb744c6208 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:30.981275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bbadbb10-aef5-477b-8835-432241072d9d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:30.989233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:30.990187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 01 06:55:30.991012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:30.993152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 173/354] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:30.994728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e8833-2648-4c64-8132-8d5debf0ddc0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:30.995217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e8833-2648-4c64-8132-8d5debf0ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:30.995702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e8833-2648-4c64-8132-8d5debf0ddc0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:30.997140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 174/355] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:30.998038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e8833-2648-4c64-8132-8d5debf0ddc0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.017633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.018814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.019246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.019834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.103316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.105467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.130099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e8833-2648-4c64-8132-8d5debf0ddc0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.165490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.172681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.174065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.174065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.174674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.176062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.178415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80187e80-5f71-445a-becb-a50bee4277d5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.180785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 175/356] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.182659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.183002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.183369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.203377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:31.204731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:31.222640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.234254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 182/357] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:31.235419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.312900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 176/358] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:31.314036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.373102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56765df3-96f7-4939-ab07-e911ff2d430f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.417815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.418496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.418797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.419094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.419852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:31.420159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.421551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.432480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 183/359] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.433939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.445162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb60d-8c7e-48b7-9bff-3a2c20b9b8ad None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.483863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.492689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.493774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.494076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.494324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.495583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.497282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a89f6-c386-4c1b-bdca-9ae25ecfb307 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.499508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 177/360] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.500899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b81412-f83d-4939-9c0c-87bdf55e2797 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.501330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b81412-f83d-4939-9c0c-87bdf55e2797 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.501836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b81412-f83d-4939-9c0c-87bdf55e2797 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.503316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 178/361] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:31.504247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b81412-f83d-4939-9c0c-87bdf55e2797 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.507928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.567297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fd960-5a68-4ce7-aff5-d8169a129772 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.610154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.611080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.611367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.611768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.612970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.614426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.616489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 184/362] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.617277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.655395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b81412-f83d-4939-9c0c-87bdf55e2797 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.695081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.695890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.696185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.696436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.708713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:31.717758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:31.719812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 179/363] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:31.720872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.755100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1ac4d-c6ed-4d76-8ed5-840784c98b7a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.780962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.789434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:31.803917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.804798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.805083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.805358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] PATH_INFO: `/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.806583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] RBAC: Authorizing `identity:get_user(user_id=2ccb7f1d1f2c4101ac99c0e06cf6cba3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.808314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382783-7bfb-48b4-b487-911fe9d8a604 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.814941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 185/364] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/users/2ccb7f1d1f2c4101ac99c0e06cf6cba3 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.816213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.816610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.816791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.836487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:31.837664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:31.844838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ca330-35c3-48fe-a2de-dfc102b99923 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:31.888203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:31.888965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:31.889340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:31.889629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:31.890466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=5d1d33a09722400a9530df3eab62138c) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:31.890799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:31.892245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:31.900003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 180/365] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=1dfd6d1514104a199766879261c434f7 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:31.900625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:31.926496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 186/366] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:31.927469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.077300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353a393-65a5-477f-8ced-b09699416a58 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.094946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e890f30-3304-4726-b515-239fc74b0610 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.113013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:32.122401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:32.143710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.144592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.144875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.145151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.147226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.148689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.151181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 181/367] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:32.152159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.183236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.184024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.184317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.184598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.185357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:32.185668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.191389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.200460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 187/368] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:32.201299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.299271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197f2-ebb4-4b52-a11f-a7c93a5c6531 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.346387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.347013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.347862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.347862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.348381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] RBAC: Adding query filter params (role.id=96d3a4c0e9534751998d64755241c000, user.id=2ccb7f1d1f2c4101ac99c0e06cf6cba3, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:32.348690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.350112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.363031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 182/369] 10.222.0.48 () {56 vars in 1308 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/role_assignments?user.id=2ccb7f1d1f2c4101ac99c0e06cf6cba3&scope.system=all&role.id=96d3a4c0e9534751998d64755241c000 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:32.363622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.423503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:32.484968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12933-1290-4d42-8553-f41d4bc2d161 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.485195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d467e-4198-4d27-ba4c-390c8086b405 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.532865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.534822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.538217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.538793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.539948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=fa7b086c98dd452db4945dd06282fd24) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:32.541712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.543639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.556790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 183/370] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=41985665942d4bba9f8355bf503f068b => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:32.559122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211578-8407-4887-b56e-304d7321f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.569601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211578-8407-4887-b56e-304d7321f896 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.570105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211578-8407-4887-b56e-304d7321f896 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.573705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211578-8407-4887-b56e-304d7321f896 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.596075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211578-8407-4887-b56e-304d7321f896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-26211578-8407-4887-b56e-304d7321f896 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:32.608881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:32.609886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:32.610701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:32.617772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 188/371] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:32.625854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211578-8407-4887-b56e-304d7321f896 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:32.697426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:32.760846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12033e-8717-43cf-9361-30838d8d049a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.777758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:32.812064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211578-8407-4887-b56e-304d7321f896 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:32.812723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.813459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.813692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.813971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.908139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:32.908754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:32.909044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:32.909329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:32.912172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:32.912172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.912172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.920062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:32.922417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:32.929332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 189/372] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:32.935182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.124410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 184/373] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:33.126155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.140770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa8414-97a5-4f38-890d-e1dd2009a606 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.142574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.147017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.201300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.203712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.204976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.206152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.299386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787a2e7-5a2a-46a3-9577-dce23ff3629a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.333621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.335164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b461a-8399-4650-a132-eba8f0eb8e7c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.344080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.345642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.346099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.346965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.348598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.350970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e05c5d-2d7e-4caa-8b60-ba4834171edc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.353424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 185/374] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:33.355356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e15fb-223e-4ab8-af2c-072be9fb45ff None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.355815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e15fb-223e-4ab8-af2c-072be9fb45ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.356280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e15fb-223e-4ab8-af2c-072be9fb45ff None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.359015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 186/375] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:33.360036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e15fb-223e-4ab8-af2c-072be9fb45ff None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.458647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 190/376] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:55:33.460413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.460413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.460666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.473712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.480027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.487964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:33.489535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:33.502803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e15fb-223e-4ab8-af2c-072be9fb45ff None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.546428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.547408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.547685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.547956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.549152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.550567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.552795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 187/377] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:33.553568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.580844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 191/378] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:33.581994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.707408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bddf8e-e638-4f20-8329-5ceb77e14cf0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.737383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58804b7-c515-4471-810e-24bfabc3baec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.746547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.747791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.747791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.748043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] PATH_INFO: `/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.749239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] RBAC: Authorizing `identity:get_project(project_id=b0142643ea3d4627a8dc9edbdfab9d31)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.750801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.752782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 188/379] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31 => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:33.759322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.775842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.779227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:33.784325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.786030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.786724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.787191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.788437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=fa7b086c98dd452db4945dd06282fd24) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:33.788895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.791182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.799479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 192/380] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=1dfd6d1514104a199766879261c434f7 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:33.800630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.929614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93305f7e-03cf-4dcd-92f4-43c074025bfe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.937969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe30cf-f086-43c7-85a5-7eced672074f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:33.970620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.971859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.972287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.972715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] PATH_INFO: `/v3/groups/747a75e666834e1b819bac374b549b00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:33.980163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] RBAC: Authorizing `identity:get_group(group_id=747a75e666834e1b819bac374b549b00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:33.982048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:33.984350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 193/381] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/groups/747a75e666834e1b819bac374b549b00 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:33.989413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:33.990459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:33.993431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:33.995084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:33.996303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.022906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:34.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:34.041142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:34.042123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:34.044291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 189/382] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:34.050091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:34.091517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:34.140698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc9226-6d95-47a5-a2b2-f51da4b554ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.157671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:34.178214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.179039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.179424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.179824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.180740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] RBAC: Adding query filter params (group.id=747a75e666834e1b819bac374b549b00, role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=b0142643ea3d4627a8dc9edbdfab9d31) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:34.181229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:34.182959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:34.188437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d1875-a50c-4299-be07-dfe09c32192a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.198995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 194/383] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:33 2022] GET /identity/v3/role_assignments?group.id=747a75e666834e1b819bac374b549b00&scope.project.id=b0142643ea3d4627a8dc9edbdfab9d31&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:34.231315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.232281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.232729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.233212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.234121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:34.234586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:34.236164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:34.246757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 190/384] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:34.253532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:34.388313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3a969-26eb-41bb-b4b6-5821b0b79799 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.412851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:34.424073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.425274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.425879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.426403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.427817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:34.429318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e969-3f1e-4ace-a381-55f0a950e6e2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:34.431262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 195/385] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:34 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:34.437633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:34.511023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38fc-8e5e-4995-9ee9-3cf161012d7e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.511306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38fc-8e5e-4995-9ee9-3cf161012d7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.511662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38fc-8e5e-4995-9ee9-3cf161012d7e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.513340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 196/386] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:34.519336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.519620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.519897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.560165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:34.561976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:34.567198 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbcfed-5757-4150-91ea-9ba9ba948d0d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.608947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.610259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.610792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.611293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.612977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:34.614834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:34.617067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 191/387] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:34 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:34.622880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:34.636965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:34.667661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 197/388] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:34.696776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:34.783095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f84d22-4ab6-4823-b18c-6e831b69b7d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.834830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa34c46-e274-4547-a7ae-e514b50f8af0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:34.840357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.841261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.841546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.841824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/groups/063a643599be43d3ae8a535d240bb85e/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.844041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, group_id=063a643599be43d3ae8a535d240bb85e, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:34.845489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:34.859256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:34.875060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:34.875919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:34.876199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:34.878697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:34.879389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 192/389] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Jul 1 06:55:34 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/groups/063a643599be43d3ae8a535d240bb85e/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:34.891625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:34.905444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:34.906414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:34.907231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:34.907854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 198/390] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:34.913012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:35.001014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:35.043866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d191419-d7b1-4f03-b3b2-db1041b34540 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:35.087516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:35.088202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.088482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.088754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.089509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:35.089811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:35.091199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:35.100576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 193/391] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:35.105695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:35.225601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0cbd2-bf25-4053-877b-961e5f61704c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:35.263159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:35.264018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.264849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.265186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.266405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:35.267907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:35.269561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 199/392] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:35 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:35.275452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:35.400395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7e732-c456-4c89-aac0-48078ac678e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:35.433679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:35.442916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:35.443782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.444057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.444325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.445529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:35.446976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6f697-fe34-442d-b849-06bcd774c62d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:35.448835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 194/393] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:35 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:35.455017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:35.466471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067b4c-4918-4183-9e2b-e9c3571f7a76 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.466757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067b4c-4918-4183-9e2b-e9c3571f7a76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.467059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27067b4c-4918-4183-9e2b-e9c3571f7a76 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.473478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 195/394] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:35.478784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.479131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.479416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.506140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:35.507803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:35.548464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:35.596125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 196/395] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:35.628153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679bb19-6b75-4f94-a779-db6f7856bd1a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:35.629721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:35.679945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:35.680758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.681041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.681320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c/groups/063a643599be43d3ae8a535d240bb85e/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.688653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] RBAC: Authorizing `identity:create_grant(project_id=5d1d33a09722400a9530df3eab62138c, group_id=063a643599be43d3ae8a535d240bb85e, role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:35.690112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:35.726132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 200/396] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Jul 1 06:55:35 2022] PUT /identity/v3/projects/5d1d33a09722400a9530df3eab62138c/groups/063a643599be43d3ae8a535d240bb85e/roles/1dfd6d1514104a199766879261c434f7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:35.798629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38181877-acc5-4912-a0ee-8e9e9a7757e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:35.828779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:35.842368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:35.843355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:35.843649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:35.843937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:35.863647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:35.871998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:35.872918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:35.874244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 197/397] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:35.879894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:36.015474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db896e-584d-4e22-b87c-538788b1c69c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:36.028750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:36.071766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:36.072452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.072743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.073023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.073786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:36.074099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:36.075506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:36.092071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 201/398] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:36.097836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:36.261908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d59b-7e03-460a-885d-95027d261b2e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:36.314610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:36.315475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.315680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.315961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.319461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:36.321133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:36.323158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 198/399] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:36 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:36.328798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:36.500128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896919b-97f0-4a5e-a7a2-17bfc8853703 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:36.549150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:36.550396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.550947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.551421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.552853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:36.554549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb2611a-5836-4cab-86c5-0d51c8e5bb5f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:36.556554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 202/400] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:36 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:36.562498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:36.641437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:36.690748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aca85c-6465-4a13-8ff3-cb9f9feb100a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:36.726248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:36.727219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.727505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.727780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/groups/063a643599be43d3ae8a535d240bb85e/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.729749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, group_id=063a643599be43d3ae8a535d240bb85e, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:36.731215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:36.739332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853b42f-07cc-40c9-9b07-cc06cb09695b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.739615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853b42f-07cc-40c9-9b07-cc06cb09695b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.739897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853b42f-07cc-40c9-9b07-cc06cb09695b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.741551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 203/401] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:36.746793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e152-5168-4cb0-abd4-5e71ac12d6ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.747303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e152-5168-4cb0-abd4-5e71ac12d6ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.747594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85e152-5168-4cb0-abd4-5e71ac12d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.750187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 199/402] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Jul 1 06:55:36 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/groups/063a643599be43d3ae8a535d240bb85e/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:36.772760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a85e152-5168-4cb0-abd4-5e71ac12d6ac None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:36.774070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a85e152-5168-4cb0-abd4-5e71ac12d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:36.813132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:36.856913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 204/403] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:36.893380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:36.950662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a350c-f39f-4e69-b4df-e0ef43f0bb5c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.950662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a350c-f39f-4e69-b4df-e0ef43f0bb5c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.950662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a350c-f39f-4e69-b4df-e0ef43f0bb5c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.950662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 205/404] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:55:36.962791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:36.962791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:36.962791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:36.981330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:36.982540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:37.018960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1775c5a-1133-4484-bebd-38f405dfa922 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.034407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:37.058459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.060324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.060606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.062203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.083451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:37.102370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:37.103324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:37.104111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:37.105447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 200/405] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:37.111329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.128251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 206/406] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:37.158122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.278137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:37.281849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4551f-f13c-4aeb-a0df-0998c9b03cf1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.323473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb6ce8-6562-4031-8962-f4b27338b086 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.328184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.328994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.329479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.329553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.352257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:37.360181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:37.361151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:37.361986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:37.373084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 207/407] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:37.376557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.398738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.399438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.399716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.399989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.400740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:37.401043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:37.402415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:37.423222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 201/408] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:37.428104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.506354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:37.549059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab1d24-6cd5-4741-bc11-a3b8ceb67027 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.618968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.620014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.620499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.621000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.621961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:37.622466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:37.625655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:37.630355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d803f0-b06c-4692-8ac1-4b0037df7be1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.644634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 208/409] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:37.655800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.678872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.678872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.678872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.678872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.679235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:37.679235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:37.679235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 202/410] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:37.684666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.713402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:37.824568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34f7e4-a57f-4123-8d63-b4cecf1769e9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.848792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:37.852324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8752-ddbb-483d-b0f6-818204ea2281 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:37.863394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.864638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.865124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.865634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.867226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:37.868902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149a84d-8b4e-44ca-b8b3-0eff32bdddc6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:37.872118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 203/411] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:37.873525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115f2b-b0ab-4acd-a099-290d12021054 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.874811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115f2b-b0ab-4acd-a099-290d12021054 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.875415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115f2b-b0ab-4acd-a099-290d12021054 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.877120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 204/412] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:37.879566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115f2b-b0ab-4acd-a099-290d12021054 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:37.901829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:37.903170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.903666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.904139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] PATH_INFO: `/v3/groups/747a75e666834e1b819bac374b549b00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.906227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] RBAC: Authorizing `identity:get_group(group_id=747a75e666834e1b819bac374b549b00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:37.908063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a3e9d-4572-436c-8444-d975dc71de5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:37.910272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 209/413] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:37 2022] GET /identity/v3/groups/747a75e666834e1b819bac374b549b00 => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:37.911717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:37.912589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:37.913361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:37.934756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:37.936536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:37.998029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115f2b-b0ab-4acd-a099-290d12021054 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.024184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 210/414] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:38.026109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.034113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.035021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.035303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.035579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24/groups/063a643599be43d3ae8a535d240bb85e/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.037883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa7b086c98dd452db4945dd06282fd24, group_id=063a643599be43d3ae8a535d240bb85e, role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:38.039438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:38.055611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 205/415] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Jul 1 06:55:37 2022] PUT /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24/groups/063a643599be43d3ae8a535d240bb85e/roles/1dfd6d1514104a199766879261c434f7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:38.060357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.062208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.148415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.166315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23749c9-2ea1-4d2b-a2a3-efdbb591fc2b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.223893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.225792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.226341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.226863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] PATH_INFO: `/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.228413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] RBAC: Authorizing `identity:get_project(project_id=b0142643ea3d4627a8dc9edbdfab9d31)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:38.230124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:38.232901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 211/416] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:38 2022] GET /identity/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:38.243811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.286687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e82a66-aac5-4a3f-b7af-d0e0b0669a96 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.323796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.324639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.324878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.325149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.340802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:38.347776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:38.348646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:38.349477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:38.350076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 206/417] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:38.354552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.413347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.426362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af388aef-44a6-454b-a204-6a7f68041d46 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.461551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.462487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.462703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.463021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] PATH_INFO: `/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31/groups/747a75e666834e1b819bac374b549b00/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.464977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0142643ea3d4627a8dc9edbdfab9d31, group_id=747a75e666834e1b819bac374b549b00, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:38.466410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e9af6-60c9-4b7b-b30c-85214d6d4af1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:38.482349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 212/418] 10.222.0.48 () {58 vars in 1442 bytes} [Fri Jul 1 06:55:38 2022] PUT /identity/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31/groups/747a75e666834e1b819bac374b549b00/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:38.487159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be46b75-0948-4d04-a01e-9f6d2eceac28 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.487462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be46b75-0948-4d04-a01e-9f6d2eceac28 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.487806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be46b75-0948-4d04-a01e-9f6d2eceac28 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.489671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 213/419] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:38.499991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.500303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.500563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.522544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.534099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628a25-876c-4b67-bb30-aa370948b544 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.551743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:38.552959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:38.580807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.581598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.581964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.582363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.583485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:38.583902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:38.585455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:38.600740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 207/420] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:38.606722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.695767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 214/421] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:38.729429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:38.767852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024e812-97a1-44d5-9ca1-7531921aed3b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.813739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.814391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.814671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.814967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.831545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.843674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:38.888964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c33dc-870f-40e1-8a6f-8d112038e871 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:38.890273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:38.891959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:38.991566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:38.992359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:38.992583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:38.992865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:38.996612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 208/422] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:39.002437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.018293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:39.032681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:39.033755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:39.034740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:39.042571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 215/423] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:39.049045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.109255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.198699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16403cc8-ffcc-405d-afb5-33d9ed02e8aa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.205897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137aacb-322e-4b8c-9e99-27bd87526b1d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.248105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.249837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.250119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.250391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.251617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.253054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb837f5-443f-474e-87d4-af0229dfc801 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.255101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.255243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 209/424] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:39.256125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.256421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212227c-7d0e-43f8-81f4-76a4b5456e6f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.256708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212227c-7d0e-43f8-81f4-76a4b5456e6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.256983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212227c-7d0e-43f8-81f4-76a4b5456e6f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.257225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.257690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.258447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 210/425] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:39.258843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:39.259288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.260799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.263783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.263930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.264213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.267789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.279364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 216/426] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:39.279798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.285893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:39.287160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:39.378875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 211/427] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:39.380537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.409720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa985dbe-b5a1-440e-94c8-705f57795625 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.449617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.450096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.452698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.453124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.453844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.456453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.458729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.461448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 217/428] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:39.462423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.508864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7defd97-f204-4c89-9e24-d23aa2389dd7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.548450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.550892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.551827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.552299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.552761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.591567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394b08-edf9-49d8-80a2-886f43ed906a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.622399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.624233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.631278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.632065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.632624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.632624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.643660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:39.652047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:39.652946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 01 06:55:39.653752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:39.658947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 218/429] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:39.659771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.680352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.729477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 212/430] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:39.730564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.772752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c92edb-cf97-4364-9d7f-fe71a34eb279 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.801964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:39.812516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.813175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.813461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.813740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.814578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=5d1d33a09722400a9530df3eab62138c) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:39.814943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.816391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.823007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 219/431] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=41985665942d4bba9f8355bf503f068b => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:39.823889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.839898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1498c2b-adb2-4c25-af16-ca544461d7f3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:39.922658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:39.923535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:39.923876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:39.924156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:39.924931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:39.925243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:39.926648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:39.950271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 213/432] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:39.965160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:39.993095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.009612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f649-f3c3-493f-bc09-57b33ac53187 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.045274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.046093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.046377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.046656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] PATH_INFO: `/v3/projects/5d1d33a09722400a9530df3eab62138c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.047904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] RBAC: Authorizing `identity:get_project(project_id=5d1d33a09722400a9530df3eab62138c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.049361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.052491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 220/433] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/projects/5d1d33a09722400a9530df3eab62138c => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:40.060830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.100967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3f57b-dabe-45f1-9605-8b6f6bc6f746 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.123197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.137896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.138695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.139140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.140116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.204978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bb19e-402b-4e7a-aeee-62dc9a766280 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.216273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.217286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.234701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.257878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.260142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.260431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.260712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.265934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.267430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.269424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 221/434] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:40.277029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.351395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 214/435] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:55:40.359213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.397674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f18c0-e7b2-4558-9bca-e20b9903b7dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.429386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.443522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.444184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.444470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.444802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.445693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=5d1d33a09722400a9530df3eab62138c) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:40.446011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.448859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9ef2f-7ed7-4c42-b5d9-b77a7de1e398 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.454784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 222/436] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=5d1d33a09722400a9530df3eab62138c&role.id=1dfd6d1514104a199766879261c434f7 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:40.456061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb975a0-dc81-446c-8831-396c8328ecca None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.456348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb975a0-dc81-446c-8831-396c8328ecca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.456626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb975a0-dc81-446c-8831-396c8328ecca None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.461827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 223/437] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:40.463828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.463828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.464007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.491143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:40.491928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d8177-59ae-4323-8533-10c6bcb3e4ab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.493349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:40.509871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.540453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.541703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.542182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.542649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.544771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.547058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.550880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 215/438] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:40.556883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.599029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 224/439] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:40.627888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.704602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6bafb-1d36-4d3d-b60f-aa030ba12694 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.755098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.756354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.756586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6a144-4f7d-4eb0-a3b1-7a39314049f7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.757053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.757333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.758670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.760935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.762900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 216/440] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:40.770227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.784569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.795503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.796559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.796983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.797396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.810649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:40.819112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:40.820143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:40.820962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:40.821968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 225/441] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:40.826666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:40.828268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:40.895287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06e886-28cc-4423-97c2-087e5c164d17 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.933089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.933734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.934015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.934286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.935142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=41985665942d4bba9f8355bf503f068b, scope.project.id=fa7b086c98dd452db4945dd06282fd24) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:40.935450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.937394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93e1b5-0314-41c0-95b6-f1cae2a8dd81 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:40.943087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 217/442] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=41985665942d4bba9f8355bf503f068b => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:40.944314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841402ae-d763-4668-9746-55f06b46542b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.944603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841402ae-d763-4668-9746-55f06b46542b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.944877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841402ae-d763-4668-9746-55f06b46542b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.946268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 218/443] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:40.948426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704085d-59a0-4ef0-a9e1-15195dce5855 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:40.952574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.952845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.953060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.970391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:40.971649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:40.988775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:40.989456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:40.989737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:40.990037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:40.990850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:40.991161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:40.992574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.003364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 226/444] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:41.009297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.049817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 219/445] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:41.067120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.073949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.132461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.133950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2238a-3449-4c17-87e1-2ac30f34b85e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.176071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.176645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.176924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.177201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.201741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9f5f1-3563-42c1-913d-d001e559c376 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.252112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.252945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.253281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.254598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.260770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.262236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.269896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:41.278324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:41.280600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:41.281505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 220/446] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:41.287509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.317458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.371280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 227/447] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:41.377334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.393519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092abe4-caf3-4786-977d-b28226fbc83c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.395807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.428686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.429268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.429548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.429823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.430561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:41.430899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.432519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.441096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 221/448] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:41.446564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.475559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa60e-030e-40f0-a7b6-63456702367c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.511862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.512705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.513000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.513255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] PATH_INFO: `/v3/projects/fa7b086c98dd452db4945dd06282fd24` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.514463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] RBAC: Authorizing `identity:get_project(project_id=fa7b086c98dd452db4945dd06282fd24)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.515953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.517639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 228/449] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/projects/fa7b086c98dd452db4945dd06282fd24 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:41.522955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.536473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.544216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5905ab-df09-4f98-8b2a-f4c68b7a91c7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.577499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.578116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.578401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.578677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.611619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.622741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c51946-5d19-4ffe-8924-a8bad15b1f3a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.645275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.646737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.656348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.657383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.657657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.657933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] PATH_INFO: `/v3/groups/063a643599be43d3ae8a535d240bb85e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.659121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] RBAC: Authorizing `identity:get_group(group_id=063a643599be43d3ae8a535d240bb85e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.660547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.662152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 229/450] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/groups/063a643599be43d3ae8a535d240bb85e => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:41.667847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.744042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 222/451] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 01 06:55:41.746885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.749640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:41.766177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf062182-b7ca-4f01-9054-cfdf721f1a1b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.799812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.800461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.800745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.801022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.801847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] RBAC: Adding query filter params (group.id=063a643599be43d3ae8a535d240bb85e, role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=fa7b086c98dd452db4945dd06282fd24) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:41.802161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.803637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.809597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 230/452] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/role_assignments?group.id=063a643599be43d3ae8a535d240bb85e&scope.project.id=fa7b086c98dd452db4945dd06282fd24&role.id=1dfd6d1514104a199766879261c434f7 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:41.811562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:41.849020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b343bc7-8d28-4977-b32c-51648a5c65b5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:41.885202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:41.886063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:41.886347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:41.886624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] PATH_INFO: `/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:41.887905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] RBAC: Authorizing `identity:get_project(project_id=b0142643ea3d4627a8dc9edbdfab9d31)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:41.889952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:41.891777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 223/453] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/projects/b0142643ea3d4627a8dc9edbdfab9d31 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:41.897375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:42.002999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52940559-b79f-414b-a4a1-1eeb2d4e38b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:42.035868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:42.036728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:42.037011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:42.037283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] PATH_INFO: `/v3/groups/747a75e666834e1b819bac374b549b00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:42.038445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] RBAC: Authorizing `identity:get_group(group_id=747a75e666834e1b819bac374b549b00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:42.039938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b35673-182c-4fa2-b590-085cb571342f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:42.041567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 231/454] 10.222.0.48 () {56 vars in 1180 bytes} [Fri Jul 1 06:55:41 2022] GET /identity/v3/groups/747a75e666834e1b819bac374b549b00 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:42.047941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:42.149629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be405f4b-3d97-44f0-8e0a-32b85b0e8f38 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:42.182679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:42.183350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:42.183634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:42.183909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:42.184731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] RBAC: Adding query filter params (group.id=747a75e666834e1b819bac374b549b00, role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=b0142643ea3d4627a8dc9edbdfab9d31) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:42.185040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:42.187088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62584fb8-c245-4d2e-898c-c441e9e75373 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:42.188861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:42.194045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 224/455] 10.222.0.48 () {56 vars in 1376 bytes} [Fri Jul 1 06:55:42 2022] GET /identity/v3/role_assignments?group.id=747a75e666834e1b819bac374b549b00&scope.project.id=b0142643ea3d4627a8dc9edbdfab9d31&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:44.567575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecde7b-75e5-495b-bbc8-84014b1dcfe4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:44.568055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecde7b-75e5-495b-bbc8-84014b1dcfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:44.568055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecde7b-75e5-495b-bbc8-84014b1dcfe4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:44.569682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 232/456] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:44.574532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482ef5-0ba5-4533-8c69-68bcbc5194ef None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:44.574980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482ef5-0ba5-4533-8c69-68bcbc5194ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:44.575165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482ef5-0ba5-4533-8c69-68bcbc5194ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:44.599757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-86482ef5-0ba5-4533-8c69-68bcbc5194ef None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:44.600982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-86482ef5-0ba5-4533-8c69-68bcbc5194ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:44.685208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 225/457] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:44.710056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecde7b-75e5-495b-bbc8-84014b1dcfe4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:44.732656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:44.743221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdeef-fd61-48f2-8d25-e894fdb5ffa1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:44.743528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdeef-fd61-48f2-8d25-e894fdb5ffa1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:44.743997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fdeef-fd61-48f2-8d25-e894fdb5ffa1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:44.745323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 226/458] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:44.754456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:44.754456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:44.754456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:44.773769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:44.775288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:44.859214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 227/459] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:44.880644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:44.904407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecde7b-75e5-495b-bbc8-84014b1dcfe4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:44.946066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:44.971601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:44.972396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:44.972675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:44.972947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:44.990591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3cc13-c772-4a3f-bac0-f9dc5ce623b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.001292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.002742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.026650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.027831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.028308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.028773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:45.030319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:45.031190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:45.032037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:45.032037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:45.032037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.032037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:45.043117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.044063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 233/460] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:45.045377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.051816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:45.077327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:45.078214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:45.079495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:45.079495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:45.079495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.079495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:45.079495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 228/461] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:45.086247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.169202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.240127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631cacac-3223-426b-ba2d-a1e9b2756bfd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.245956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef7840-a6b0-4201-afec-3ccece17220d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.262463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.293625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.294289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.294575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.294891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.296132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:45.296441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.297811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9301a40c-0f4b-4f1a-919c-b4cd7eee8154 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.314822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 229/462] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:45.314822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49006a-6b16-427e-96e0-40ac387fdc2a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.314822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49006a-6b16-427e-96e0-40ac387fdc2a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.314822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49006a-6b16-427e-96e0-40ac387fdc2a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.316247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.317166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.317852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 230/463] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:45.317966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.318417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.319021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.319313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.319592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.319975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:45.320465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.322344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.336549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:45.337417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 234/464] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:45.337745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:45.338791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.414943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 231/465] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:45.414943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.456345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617af78-2e8f-4c84-9a98-c17bf737ba6e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.480994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.494005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.495203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.495203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.495410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.496055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.497491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876630e-d774-4d9b-8828-5f1e891b69ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 235/466] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:45 2022] POST /identity/v3/users => generated 248 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9870037-85e8-42d3-bf26-c57213350d01 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9870037-85e8-42d3-bf26-c57213350d01 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9870037-85e8-42d3-bf26-c57213350d01 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 236/467] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:45.526689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9870037-85e8-42d3-bf26-c57213350d01 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.537206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.540949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52ae1e-f64e-4dd4-84d4-d45003875200 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.585770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.586799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.586936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.587283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.587900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.589545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe583145-b149-4f9d-af39-7dc072ecacf3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.614589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 232/468] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:45 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:45.615886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.616250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.616534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.635823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:45.637213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:45.659390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9870037-85e8-42d3-bf26-c57213350d01 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.699931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.700931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.701216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.701490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.714007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.715723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.720853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 233/469] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:45.751206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:45.752124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:45.752993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:45.752993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:45.752993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.752993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:45.752993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 237/470] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:45.760271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.771557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.807107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.869874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:45.891932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dff4b0-617b-47e2-a79b-aaaa8f85857e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.892870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34cad2-879e-4bba-b110-2aaaeae93d84 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:45.944153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.944396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:45.944827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.945112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.945393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.945774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:45.946131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:45.946402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:45.946508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.946967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:45.947863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.959042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:45.960667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:45.966200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 234/471] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:45.973352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:45.984066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:45.984948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:45.985815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:45.985815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:45.985815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:45.985815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:45.985815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 238/472] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:45.991552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:46.088997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c2e9-bef8-43af-80ce-e872fc1eeaea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.120503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:46.130796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.131468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.131750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.132017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.132724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:46.134149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663c43-a8d7-4a54-804e-1b62a285a543 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:46.143428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b39b6-db86-4764-b095-e0e6a75e719e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.159166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 235/473] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:45 2022] POST /identity/v3/users => generated 249 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:46.160599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689320-97c6-4414-83f7-de5d839726dc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.160890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689320-97c6-4414-83f7-de5d839726dc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.161167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e689320-97c6-4414-83f7-de5d839726dc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.162988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 236/474] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:46.168403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.168678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.168962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.175233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:46.189849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:46.191956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:46.214519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.215236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.215514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.215783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.216587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:46.216832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:46.218325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:46.238326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 239/475] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:46.245011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:46.289825 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 237/476] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:46.313605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:46.416556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0e26-70fd-4fc9-9e20-c2bbfda280b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.419200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd5583-ce10-418a-a518-1a1fc9a78cee None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.454102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:46.459576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.460252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.460618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.461145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.461250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.461606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.461724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.461883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.462530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:46.462633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:46.464023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:46.464230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:46.472674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 06:55:46.472674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 06:55:46.472674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [parameters: {'id': '7b0951f556fd42b387ea5eaca46a25b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 06:55:46.472674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 06:55:46.482054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:46.490057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 240/477] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:46 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 01 06:55:46.494916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return ref.to_dict() Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:55:46.495849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 06:55:46.496796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 06:55:46.497679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:55:46.499402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:55:46.501094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'id': '7b0951f556fd42b387ea5eaca46a25b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:46.502538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 01 06:55:46.503744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:46.505049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 238/478] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Jul 1 06:55:46 2022] POST /identity/v3/roles => generated 144 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:46.506197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:46.634441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a35e9-2f19-4087-99f3-3c63153c36b2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.673577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.675184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.675748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.676218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.687802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d4f50220-0937-44b0-bb34-e460897e79cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:46.696186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:46.697052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 01 06:55:46.697855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:46.700517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 241/479] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:55:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:46.706356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:46.821209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d6b8f-6543-4634-b5ba-f0933844dd56 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:46.855843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:46.856855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:46.857329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:46.857794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:46.859287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:46.859809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:46.861384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363025f6-bc76-4b6e-af22-e7ea64c78fd0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:46.870223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 239/480] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:48.281144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d3bd-17dc-4bdf-9ab7-01436ece5e98 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.281648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d3bd-17dc-4bdf-9ab7-01436ece5e98 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.281746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d3bd-17dc-4bdf-9ab7-01436ece5e98 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.283323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 242/481] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:48.288511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.288892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.289072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.313150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:48.314340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:48.357086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26acac-be3e-48af-bba2-3e2f25d23df2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.357383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26acac-be3e-48af-bba2-3e2f25d23df2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.357660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26acac-be3e-48af-bba2-3e2f25d23df2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.359356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 243/482] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:48.364119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.364413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.364693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.386335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:48.394254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:48.399219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:48.402577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 240/483] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:48.427706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:48.533735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:48.537970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958af555-91c3-4673-84b6-986d1baa7c7f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:48.561911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 244/484] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:48.585639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:48.586494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.586800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.587084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.603125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:48.617274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:48.618205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:48.619415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:48.620305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:48.620388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 241/485] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:48.624812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:48.675020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:48.729008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f983013-5d21-4b3c-8145-63a623a3c01e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:48.764246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:48.764910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.765187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.765462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.766202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:48.766505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:48.768013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:48.776397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 242/486] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:48.781688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:48.828476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c46f4-eeb0-4f00-aa09-37439d08bbb3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:48.864624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:48.865422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.865707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.866086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.877147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:48.885411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:48.887687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:48.890168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 245/487] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:48.901597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:48.919495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b73c53-8cc4-4fbf-bdad-a183ffb4acbf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:48.931666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:48.958089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:48.958986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:48.959270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:48.959546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:48.971820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:48.973313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eb96a45d-4ee6-4631-aa7a-c5b287702f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:48.993187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:48.995333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:48.997445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:48.997445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:48.997445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:48.997445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:48.997445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 243/488] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:49.000451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1b21-6116-4a74-9d5f-cf7a6674e5ef None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.000738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1b21-6116-4a74-9d5f-cf7a6674e5ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.001018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea1b21-6116-4a74-9d5f-cf7a6674e5ef None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.002407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 244/489] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:49.003793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea1b21-6116-4a74-9d5f-cf7a6674e5ef None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.027044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.077536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa2f0d-a362-42a2-9c93-927234430fd3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.110453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea1b21-6116-4a74-9d5f-cf7a6674e5ef None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.113179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.113850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.114135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.114414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.115197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:49.115507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.116899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494de236-246f-4650-b0c8-4764f9aa99c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.126306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 246/490] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:49.127543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.127834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.128116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.145005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:49.146201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:49.154341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.155305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.155776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.156053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.156837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:49.157143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.158561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c56e4-6934-45cd-be44-c3c34ba6d059 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.175035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 245/491] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:49.175035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5c88-aed8-47b3-8f83-be19b00df0b5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.175035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5c88-aed8-47b3-8f83-be19b00df0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.175035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5c88-aed8-47b3-8f83-be19b00df0b5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.175035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 246/492] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:49.179122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5c88-aed8-47b3-8f83-be19b00df0b5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.214595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.222554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 247/493] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:49.223582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.272869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.310659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5c88-aed8-47b3-8f83-be19b00df0b5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.334053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6361d1-58c7-4317-ac31-51318beb13e0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.347628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.348464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.348751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.349043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.359263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.360851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.370702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.371515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.371794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.372066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-59e412b4-90a4-40fb-b451-d913a2ec550c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:49.378616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:49.379464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:49.380386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:49.380386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:49.380386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:49.380386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:49.380386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 247/494] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:49.381252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.381351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.381567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.385328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.386815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.398568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:49.399806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:49.403347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:49.404258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:49.405783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 248/495] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:49.409705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.474814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 248/496] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:49.474814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.485411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.519694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb321e-4d6f-495f-b4e0-2c73f52ce083 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.521443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.556850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.557891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.558173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.558449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.568936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:49.576612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:49.577756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 249/497] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:49.578757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.591506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ac4c5-b85d-4da2-9857-7452ffb76a5c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.625548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.626136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.626423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.626705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.627453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:49.627761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.629129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.640132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 249/498] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:49.640907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.679145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d100-dc9c-46a6-9042-3b2cdf510e2b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.704059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.713302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.713965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.714251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.714531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.715411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:49.715727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.717182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae80836-21e6-4367-a259-4f1a5b33e404 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.747243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37884c2-91fd-46b7-8869-d9887f9322ae None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.757086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 250/499] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:49.758380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d34ae-99f4-4848-aa35-2f00ddf8fafb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.758757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d34ae-99f4-4848-aa35-2f00ddf8fafb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.759030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d34ae-99f4-4848-aa35-2f00ddf8fafb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.760348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 251/500] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:49.762504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d34ae-99f4-4848-aa35-2f00ddf8fafb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.781284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.782188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.782475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.782752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.784528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.792899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:49.801004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:49.802064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:49.803086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 250/501] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:49.803910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.867784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d34ae-99f4-4848-aa35-2f00ddf8fafb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.900774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.901364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.901651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.901743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6fcc7-d696-434d-92e5-9c1e445b2f4b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:49.901931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.902679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:49.903026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.904460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.912705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 252/502] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:49.913484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:49.925859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:49.936841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:49.937620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.937913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.938193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.945772 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:49.947266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-84c3db48-deaf-432d-a50b-83e30c991109 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:49.962075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:49.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:49.964402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 251/503] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:49.965086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:49.965086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:49.965086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:49.981250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:49.982430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:49.999191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.016734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fc220-17c6-42b7-b596-39ea02e0fc38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.049521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.049792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 252/504] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:50.050163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.050451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.050616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.050734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.119233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.120698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.152613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a9b5a-4db3-4056-905d-e278d8f55ef5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.166150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.186689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.187376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.187654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.187927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.188692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:50.188996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.190373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.198453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 253/505] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:50.198975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.199353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.219608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 253/506] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:50.220444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.301415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741578a-c726-483a-a3c3-a200512eda7f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.317918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3466e-785b-4572-839f-b7ed54068a83 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.335028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.335793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.336071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.336345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.346949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.348375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.351416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.352078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.352361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.352640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.353464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:50.353778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.355239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:50.364929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:50.365769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:50.366622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:50.366622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:50.366622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.366622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:50.366622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 254/507] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:50.366942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.371799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 254/508] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:50.372589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.403216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.413393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.469896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1324037-d62f-41cb-89c3-d5bf8e2a7ad6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.473841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5d21a-f446-40e7-a6b8-2fab1d38228c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.505623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.506298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.506427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.506682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.506986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.507110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.507397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.507677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.517363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.518846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.518952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.520326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:50.536508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:50.537442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:50.538287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:50.538287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:50.538287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.538287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:50.543683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:50.544550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:50.545410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:50.545410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:50.545410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.545410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:50.545627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 255/509] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:50.545627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.545744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 255/510] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:50.545744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.619334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.632089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.638561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb21dc8-d53d-4d37-87f7-3bdddb517cd3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.641380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053868d3-b089-4c82-bdf1-4ac28c7ce6a9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.671152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.671732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.672012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.672285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.672990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:50.673290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.674238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.674651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.675054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.675346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.675620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.686591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.686708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 256/511] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:50.687460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.688071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:50.705591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:50.706447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:50.707339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:50.707339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:50.707339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:50.707339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:50.707339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 256/512] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:50.707608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.790736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e67b38-00f1-477b-8238-80735b04e325 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.803189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec37869-2dde-4d91-8554-1985d6e52143 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.823857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.824484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.824761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.825032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.835673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.836952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.837572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.837857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.838119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.838857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:50.839187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.840538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.852148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 257/513] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:50.852987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:50.856258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:50.897629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.899150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.951294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bc63-fd31-465f-b1a1-a7a0a80057b6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:50.985400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:50.986061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:50.986342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:50.986613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:50.987359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:50.987663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:50.989027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:50.995656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 257/514] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:50.996483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.000997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 258/515] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.001795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.037081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.062963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.094934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e333266-e9a5-4e50-8cdf-4e0577e5f497 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.100497 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7777d04-cba0-4b4e-a13d-336e3057f9af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.128102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.128880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.129156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.129431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.133105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.133745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.134028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.134302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.135039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:51.135351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.136223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.136718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.137631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.149067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 259/516] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.149853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:51.151597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:51.152453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:55:51.154008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:51.155022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 258/517] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:51.156032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.255158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ce03e-5073-47aa-8336-0e221ef37e75 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.261300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.264674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bb1ae-c883-4d61-9278-cf30fcb0b2bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.279555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.290217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.291032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.291314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.291585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.302754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.304248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.306299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.307189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.307735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.308011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.318828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:51.319492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:51.320520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:55:51.321513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:51.322017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.322085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 260/518] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:51.322085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:51.341091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:51.342284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 259/519] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:51.343312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.424293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cc6ec-a455-43ef-b7bf-72fd219fc75d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.440229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3769e-1703-410a-97e1-1878508c41a1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.456531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.457182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.457459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.457737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.458428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.459896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.475473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.476189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.476472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.476746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.479393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 261/520] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:55:51 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:55:51.480193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.485570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.487142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:51.505305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:51.506288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:51.507618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 260/521] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:51.508092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.531200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.532921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.608663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf810c-00cb-4661-8e93-a43d74ba1507 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.639756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446320e9-fa52-43b4-9c16-70a30ad67be3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.652839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.653748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.653860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.654288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.655458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:51.655769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.657620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.677385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 262/522] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.678199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.683854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.684530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.684811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.685087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.685939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:51.686288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.688225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.726759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 261/523] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.727569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.789604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.793986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b681-c66c-4bb7-b16d-06cfddfe010e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.800129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:51.832146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.832806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.833086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.833359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.833587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994dcfc-240a-4912-8495-67ee3b38a73b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.834077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:51.834383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.835812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.852040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 263/524] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.852684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.868748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.869402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.869683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.869958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:51.870737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:51.871072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:51.872512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:51.891583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 262/525] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:51.892452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:51.960983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b198d6-62d1-4397-a563-4e9290be7355 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:51.993897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:51.994456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:51.994742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:51.995045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.005297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e0710-22ec-48b9-bed6-b438a90daed2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.029285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.033244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.040321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.040964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.041247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.041522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.073823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.075462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.113442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.115205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.180123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 264/526] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:52.180941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.235208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 263/527] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:52.236347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.251508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.267277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.297519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c1eda-5dfa-447d-883b-646b4e5d468f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.331258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.332043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.332328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.332609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.339188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.340347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867ad51-cdfb-4912-802b-2a6e7eb2fa42 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.340625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:52.355518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:52.356361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:52.357194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 265/528] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:52.358707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.382017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.382669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.382982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.383258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.384076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=f1d06e6ad60240329d83bc4767439ee5) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:52.384382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.385799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.391413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 264/529] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/role_assignments?user.id=f1d06e6ad60240329d83bc4767439ee5&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:52.392103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.485382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e398e9-d14e-4f31-8c2b-5a9315d31b82 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.505446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.526550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.527381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.528324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.528781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.529389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.538017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b53d-4736-4a42-8001-a1efe0d99b03 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.544124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.545627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:52.566253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:52.567400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 266/530] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:52.568421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.580370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.581658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.582044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.582425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.596592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.598801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:52.624955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:52.626034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:52.627131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:52.627131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:52.627131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:52.627131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:52.630891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 265/531] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:52.632065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.699546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6b50b-734a-4238-b046-d42316d56b61 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.735443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1df4b-8a9f-42d8-96bd-6fdf65d2fecf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.743297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.743954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.744235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.744509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.745657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:52.745965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.748008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.771241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.771699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 267/532] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:52.772539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.776714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.777102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.777478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.778315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:52.778719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.780922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.793798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 266/533] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:52.794798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.804019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.807739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:52.881229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a533345-f6af-4943-bf34-eaad2a3d7f38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.903662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53009013-b58d-44ea-b87c-f5eb0badfa37 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:52.916687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.917565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.917987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.918400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.919273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:52.919737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.921411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.933787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 268/534] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:52.934888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:52.942202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:52.943165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:52.943546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:52.943940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:52.944865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:52.945291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:52.946916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:52.958961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 267/535] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:52.960114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.038865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:53.051161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73022714-cb18-440e-a82a-03a3c8fcf21c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.090450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.091547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.091971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.092427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.100258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.101976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:53.117611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:53.118681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:53.119742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:53.121851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 269/536] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:53.127700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.155971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:53.222894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72aea3-000c-454e-be3f-ba2dd33926dc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.274029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.275385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.275808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.276218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.283719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.285353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:53.300768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:53.301820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:53.302850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:53.305133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 268/537] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:53.310514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.360693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572d0db-71ed-4714-9590-67a38292a754 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.421277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e4672-c7e4-45e6-bf5a-b23588ad0cb3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.451786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:53.458949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.459794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.460202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.460580 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.461471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:53.461880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.463436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.464022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.464695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.464982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.465264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.466057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:53.466370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.467867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.476499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 269/538] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:53.487755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.489037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 270/539] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:53.500241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.542492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:53.662336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f84d98-7563-4f04-a5c8-a866bdf38773 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.688660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa7b2-a115-4967-aeab-523f28394a0e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:53.699215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.700426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.700792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.701146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.702536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=58947ba37ab14f61adde0bf2f34829f9) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:53.703351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.705134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc99d03-915f-4c02-b95a-7e9ec5a5fe88 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.712590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 271/540] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:53 2022] GET /identity/v3/role_assignments?user.id=58947ba37ab14f61adde0bf2f34829f9&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:53.715519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233bdb7-92af-40b5-a86f-3341aade3031 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.715817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233bdb7-92af-40b5-a86f-3341aade3031 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.716398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233bdb7-92af-40b5-a86f-3341aade3031 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.724005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 272/541] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:53.725487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.725777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.726060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.753884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:53.754564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:53.754874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:53.755158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:53.756015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=52d7bcc4b67c495f821519996cc2ab26) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:53.756330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:53.757793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:53.763962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:53.765253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:53.767972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 270/542] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:53 2022] GET /identity/v3/role_assignments?user.id=52d7bcc4b67c495f821519996cc2ab26&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:53.845028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 273/543] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:53.869362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:53.949766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:53.999100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1319c-a4ca-4923-86eb-9d403dc49b61 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:54.034258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:54.035241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.035634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.036016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.046565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:54.054392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:54.055480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:54.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:54.058816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 271/544] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:54.064442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.067663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:54.167258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9e779-8ae2-475a-b577-2b3b91a4c957 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:54.213535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:54.214607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.215017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.215400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.216326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:54.216755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:54.222903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:54.232262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 274/545] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:54.237478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.368662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca99ee2-be59-4a01-a313-fa961930d014 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:54.383967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:54.405927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:54.406795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.407143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.407426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.419830 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:54.421380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:54.442708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:54.443656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:54.444507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:54.444507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:54.444507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:54.444507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:54.444507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 272/546] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:54.452457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.533892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:54.569572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561401c-9eb8-4632-af8f-fe5838200d50 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:54.604548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:54.605221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.605507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.605782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.606511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:54.606844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:54.608253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e9db0-9886-4467-99e3-f73cdb70795a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:54.620719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 275/547] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:54.626687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.708451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64cb11-e75e-41ea-a0a8-db973fa9f5e0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.708768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64cb11-e75e-41ea-a0a8-db973fa9f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.709009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64cb11-e75e-41ea-a0a8-db973fa9f5e0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.710381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 276/548] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:55:54.716335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.716623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.716905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.746180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:54.751541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:54.770227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d67c1-96a5-4c46-9e71-3b4dacdc2c75 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:54.806799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:54.808187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:54.808482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:54.808757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:54.817434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:54.818889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:54.834343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:54.835727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:54.836873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 273/549] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:54.843366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.875714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:54.915248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 277/550] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:55:54.983526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:54.984945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e88d-d9c2-4a05-a504-9a17bd7b315c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.044517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.045197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.046013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.046289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.047165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:55.047472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.048893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.057259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.068996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 274/551] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:55.075793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.114615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95fde1-bb61-4c62-b618-c3ae0fe52708 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.150412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.151258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.151542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.151920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.163559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:55.172204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:55.173131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:55.174163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 278/552] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:55.181810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.218709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.251510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfdf39-f81e-426e-ad36-b3c096e35caa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.288413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.289077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.289363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.289645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.365734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.367228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.405211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba055b0-c4e1-4b4b-87ea-671d4a9afded None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.438794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.455425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.473748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 275/553] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:55:55.479235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.479388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.480342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.480825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.481313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.482258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:55.482763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.484900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.501978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 279/554] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:55.507287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.619151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c92c0-bbc7-4e43-b3b3-386ecd273bdb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.628136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011fa96-1fa5-46ee-8692-64a35cad061f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.665439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.667520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.669266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.669750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.685798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.687691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.700889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.702232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.702522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.702822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:55.706994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:55.707836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:55.708687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:55.708687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:55.708687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:55.708687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:55.711883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 280/555] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:55.720081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.721238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.721527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:55.742317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:55.746812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 276/556] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:55.748217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.775674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.807115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:55.859996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2d6ce-c321-43d4-8cc2-9db4e707a83c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.880619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ec352-91e6-4589-8ef0-cf5e525766ae None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:55.895717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.896390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.896673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.896951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.897679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:55.897993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.899412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac70bff-a80d-41d3-9b73-c6b4f7d00d8c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.914820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 281/557] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:55.914820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b94b-8720-4607-8d58-14e4c351d501 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.914820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b94b-8720-4607-8d58-14e4c351d501 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.914820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b94b-8720-4607-8d58-14e4c351d501 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.914820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 282/558] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:55.916265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b94b-8720-4607-8d58-14e4c351d501 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:55.926196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:55.927181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.928012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.928526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.930084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:55.930616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:55.933119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a002e7-a4f0-4a00-92a0-ab35efa6d594 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:55.954083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 277/559] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:55:55.954083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:55.954083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:55.954083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:55.979244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:55.981329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:56.034210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.046645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b94b-8720-4607-8d58-14e4c351d501 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.058167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 278/560] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:56.059058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.086794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.087974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.088479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.089286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.089843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.099091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.100820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:56.116920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:55:56.118870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:56.121528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 283/561] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:55:56.122707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.173992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897fd31-767f-47e8-bd83-8caba6121798 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.220471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.221269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.221651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.221920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.228615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.230031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.231751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c919f7-b99d-4012-b603-8ac8d3a34064 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:56.243822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.244982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 279/562] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:56.245930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.273341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.274129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.274395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.274677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.284508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a3034671-74fd-4f43-9887-8832502cea7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:56.292165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.293080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 284/563] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:56.294513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11cf2-669e-47b8-bc86-be62f7b2906e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.295878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11cf2-669e-47b8-bc86-be62f7b2906e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.295878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a11cf2-669e-47b8-bc86-be62f7b2906e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.297012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 285/564] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:56.297907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df65e-3cff-40e1-8f5d-9d33d6bc0175 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.298172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df65e-3cff-40e1-8f5d-9d33d6bc0175 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.298457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df65e-3cff-40e1-8f5d-9d33d6bc0175 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.299930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 286/565] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:56.300641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8df65e-3cff-40e1-8f5d-9d33d6bc0175 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.313582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.336582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.351423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0baff-44fc-4790-b8b9-39725cf2e226 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.384202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.384867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.385147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.385425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.386250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:56.386558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.388014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.402906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8df65e-3cff-40e1-8f5d-9d33d6bc0175 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.404433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 280/566] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:56.405258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.441144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.441728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.442009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.442285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.443277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:56.443838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.445461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1436e8-fbc9-492e-9f59-e0024edbdb3b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.456654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 287/567] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:56.457888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962cafa-b873-4eb5-bfab-1b636794581c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.458179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962cafa-b873-4eb5-bfab-1b636794581c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.458453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962cafa-b873-4eb5-bfab-1b636794581c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.474151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b962cafa-b873-4eb5-bfab-1b636794581c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:56.475372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b962cafa-b873-4eb5-bfab-1b636794581c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:56.507531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b269dd-043e-47fb-894d-31e67b82fe4f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.536686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.541823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 288/568] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:56.543064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.543173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.543394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.543628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.543784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.544028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.544304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.545044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:56.545884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.547269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.559743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 281/569] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:56.559743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.564961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:56.566150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:56.567824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.631914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 289/570] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:56.632742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.659078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb558a26-d20d-4171-9e7a-48720a51ef03 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.691766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.692484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.692773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.693049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.703161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.704602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:56.721416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.722296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 282/571] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:56.728119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.737253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00f790-f740-4af9-b4df-058ce541b3b4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.764400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.770458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.771206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.771488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.771759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.772586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=eae0222c39364fa69be2a3cc938516bd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:56.772897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.774315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.780227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 290/572] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/role_assignments?user.id=eae0222c39364fa69be2a3cc938516bd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:56.781023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.789794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:56.829171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699a1a2-9e26-4266-8343-318536111bc3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.867755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.868556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.868840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.869114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.880832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:56.882347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:56.893996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38ad6-47e0-49e5-9120-7946902cb183 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:56.901490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.902546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.903469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:56.903469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:56.903469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:56.903469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:56.903469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 283/573] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:56.903769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:56.939907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:56.940859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:56.941148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:56.941728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:56.952439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:56.960209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:56.961326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:56.962269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 291/574] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:56.963079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.022174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be6bc5-8e40-45e1-bc1f-36acc02c8c8e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.024746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.035327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.060667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.061464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.061751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.062022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.072460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:57.074196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c6cee-a968-462b-8903-d9aa08d3af19 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:57.080738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:57.081678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:57.082512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 284/575] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:57.083851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.111409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.112071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.112354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.112633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.113358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.113666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.115146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.127842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 292/576] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.128484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.200549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f128da-bc7e-4d23-9194-025746224454 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.237935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b648ef-5f56-462b-aa5e-c4915663c126 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.241098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.241768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.242049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.242323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.243079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.243384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.244760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.252426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.258396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 285/577] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.259928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.275058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.275721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.276004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.276279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.277087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.277391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.278834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.291991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 293/578] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.291991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.299433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.376855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9ea7a-190d-47ed-a1c3-d8f4951a6722 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.400592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df2b3-3b78-410a-b00a-f5c9f3564110 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.414831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.415502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.415783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.416059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.416807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.417122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.418487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.427005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 286/579] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.427657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.440677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.441701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.442156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.442603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.449597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.451203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:57.466068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:57.467079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:57.467989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:57.473711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 294/580] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:57.473711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.510064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.527090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.537469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a25df-9c6b-4091-91b1-138603d244ba None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.572422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.573207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.573488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.573758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.581992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.583528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.589202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f974da-9f88-431d-ab8f-0d7f173b5daa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:57.597722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:57.598746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 287/581] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:57.599730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.628218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.629001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.629280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.629552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.640258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.641775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:57.658942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:57.659807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:57.660695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:57.660695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:57.660695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:57.660695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:57.660695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 295/582] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:57.661646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.717963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bdbe-c396-4b97-aae6-05e35c298046 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.752194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.752977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.753254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.753524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.756567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.764484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.765930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.769755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7ee9b-4489-4cb1-8e2e-ec5c1c744dc5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:57.783397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:57.784282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:57.785151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:57.785151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:57.785151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:57.785151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:57.785151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 288/583] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:57.785471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.794385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:57.812376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.813028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.813308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.813586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.814359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.814943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.816128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.828208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 296/584] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.828928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.902563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aefb80-8daf-410d-a652-7564e55c9dbb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.936593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c981d33-68ec-4644-9418-c4878ea32dcd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:57.941108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.941782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.942067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.942345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.943251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.943566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.944992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.962873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 289/585] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.963690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:57.976703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:57.977300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:57.977572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:57.977859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:57.978879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:57.978986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:57.980284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:57.992437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 297/586] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:57.993195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.004025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.033737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.075836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977912fe-d121-4746-a687-b83eb7d75cfa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.097321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4e4b-717f-4c99-a255-92fab019183a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.109571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.110227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.110515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.110808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.111521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:58.111824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.113189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.129818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 290/587] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:58.129818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.137918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.139323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.139616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.139896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/f1d06e6ad60240329d83bc4767439ee5/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.155463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=f1d06e6ad60240329d83bc4767439ee5, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.156885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.171634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 298/588] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:57 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/f1d06e6ad60240329d83bc4767439ee5/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:58.172435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.257585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb9198-4227-4b49-9d0b-8eea3004216d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.267683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.288100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.302415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.303504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.304024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.304445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.317453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.319360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.328741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579056-6a87-4bfa-9ae8-e390a98922b6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:58.339170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:58.340160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:58.341073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:58.341073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:58.341073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:58.341073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:58.348833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 291/589] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:58.350147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.375894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.376859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.377213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.377558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.385453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.387356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:58.406915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:58.408052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:58.409079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 299/590] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:58.409704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.474998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0036d46-9fb7-40ea-99d6-bad1e25af323 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.513426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.515547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.516420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.516881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.527941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.531707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.539718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c11395-c3a3-4ad0-922c-0cb43dd284f8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:55:58.551377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:58.552351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:55:58.553280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:58.560333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 292/591] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:58.560333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.573837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.593602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.594241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.594526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.594825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.595556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:58.595863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.597338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.609600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 300/592] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:58.610412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.610525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.682734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d166-44cc-4e4f-804e-9c49c1ee8d01 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.719352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.720034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.720287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.720563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.721401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:58.721711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.723168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.741213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 293/593] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:58.742073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.747287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b897c80-79b7-4d7d-a1e4-87d7c1d6fa38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.783794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.784382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.784660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.784929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.785822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:58.786129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.787672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.814927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 301/594] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:58.815749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.852444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.858022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:58.868616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a8b10-68a4-414f-aecf-8e87dff5c543 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.905717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.906513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.906860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.907111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.914308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.915801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.920949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fda4d-6209-4527-843a-78fc81790e1d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:58.933612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:58.934495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:58.935734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:58.938911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 294/595] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:58.939777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:58.963485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:58.964281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.964567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.964848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:58.975474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:58.976916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c1c75aa8-39f9-4943-9d77-728a74c62cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:58.996179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:58.997105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:58.997987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:58.997987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:58.997987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:58.997987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:58.997987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 302/596] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:58.999406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b1ccb-2e45-4e34-a23e-e0b2a15d7532 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:58.999615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b1ccb-2e45-4e34-a23e-e0b2a15d7532 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:58.999893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b1ccb-2e45-4e34-a23e-e0b2a15d7532 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.001297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 303/597] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:55:59.005892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b1ccb-2e45-4e34-a23e-e0b2a15d7532 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.052609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc865dd-5a7c-42ed-8b8a-3f1d18021133 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.087654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.088438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.088725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.089008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.099677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.101141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.101689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.116580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b1ccb-2e45-4e34-a23e-e0b2a15d7532 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:55:59.119076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:59.120043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:59.120988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:55:59.120988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:55:59.120988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:55:59.120988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:59.120988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 295/598] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:59.121274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.129810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.156308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.156962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.157253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.157513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.158349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:59.158594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.160055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2846f2a-3c86-4f92-9191-2ed004e8a22a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.171687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 304/599] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:59.172858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.173147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.173422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.190411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:55:59.194243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:55:59.253954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4cba0-b5f6-4bbd-abc4-1c7dc004735c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.260994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 305/600] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:55:59.270932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.304287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.304973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.305327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.305671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.306391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:59.306695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.308344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.321069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 296/601] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:59.321865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.350880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.374958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc34af3-2bb2-472c-91f9-9dd4c01aba1a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.392396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.413961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.414630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.414961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.415242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.415968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:59.416278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.417648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.429900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 306/602] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:59.430718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.440485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07270c-2ff7-411b-b1e6-2884e1bfbab8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.477822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.478713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.479031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.479308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.493993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=458dc4b52e3d4a13a8a4f801ddf2e6c4, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.495534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.509869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 297/603] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:55:59 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:55:59.510462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.557271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a23c-e9e0-41ec-a204-a8bbb5982c97 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.596779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.597665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.597957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.598237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.606632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.612666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:55:59.626802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:59.627820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:55:59.628666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:59.629312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 307/604] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:59.630424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.640657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067d92c-4d54-436b-9762-adc7b19cf98c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.661188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.691207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.691545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.691896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.692173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.703914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.705495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:59.719998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:59.720926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:55:59.721809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 298/605] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:59.722287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.758693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd0f9a-9152-482f-b2cb-e3b4be8f8016 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.803266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.804728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.805428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.805893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.815664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.817749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.827182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3dd-1e0c-48e6-aec2-2d6b5d98dba5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:55:59.835947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:55:59.836845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:55:59.837734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:55:59.840502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 308/606] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:55:59.841510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.879119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:55:59.880029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:55:59.880332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:55:59.880616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:55:59.882284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:55:59.882599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:55:59.884013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:55:59.896553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 299/607] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:55:59.897479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:55:59.916115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.946101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:55:59.966661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8732e7-1dcf-4677-936d-a904abfa2f5c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.004428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.005528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.005995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.006552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.007597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:00.008111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.009951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.024024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 309/608] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:00.025216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.026690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18984787-2fd3-4189-a8f1-ae34be46750c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.070045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.070632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.070953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.071236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.072418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:00.072737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.074626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.094802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 300/609] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:00.094802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.139363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c284ec-022f-43a7-828d-456812ac720e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.174927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.176144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.176626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.177097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.188383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.195136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.209130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.215222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.216337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313f11-2034-467e-811e-b8839bc76904 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:00.224463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:00.225451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:00.226344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:00.226344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:00.226344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:00.226344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:00.230839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 310/610] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:00.231994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.261650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.263122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.263796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.264343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/58947ba37ab14f61adde0bf2f34829f9/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=58947ba37ab14f61adde0bf2f34829f9, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.285093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.307985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 301/611] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:00 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/58947ba37ab14f61adde0bf2f34829f9/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:00.309481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.377230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1beac5-0c52-4845-b7fb-aec62f2dee9c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.414846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.415741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.416020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.416341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/52d7bcc4b67c495f821519996cc2ab26/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.433041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=52d7bcc4b67c495f821519996cc2ab26, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.434494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756e2e2-1493-4158-982c-628ee2694906 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.442193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd91567-09ed-4745-8a40-f8ae8832069b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.454784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 311/612] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:00 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/52d7bcc4b67c495f821519996cc2ab26/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:00.457516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148fda2-9af3-45b4-8c33-85950cc7e681 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.457967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148fda2-9af3-45b4-8c33-85950cc7e681 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.458661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148fda2-9af3-45b4-8c33-85950cc7e681 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.461397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 312/613] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:00.466306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.466675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.467040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.491861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.492867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.493321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.493809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.494860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:00.495431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.497099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.497221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:00.498354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:00.511311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 302/614] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:00.516202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.518376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.534585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.598792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 313/615] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:00.621147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.675126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be057-ba2c-4b29-874e-805063f58cb1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.718523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.721273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.721273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.721273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.730614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.732348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:00.753103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:00.754301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:00.755499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:00.758054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 303/616] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:00.766802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.768723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bce30-d88d-4df3-b3f1-605e5a67bcf1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.807173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.808379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.808665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.808944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.820327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:00.829098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:00.830040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:00.830900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 314/617] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:00.837588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:00.856513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.870683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:00.914325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de03c-8cea-483a-a1b8-c0dd06ff2536 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:00.977892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:00.979081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:00.980246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:00.980682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:00.981891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:00.982576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:00.984269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.009250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 304/618] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:01.017036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.054159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e534e3a-e414-402e-a208-cce7cb17145a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.112116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.112794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.113078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.115955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.116731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:01.118472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.119880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.139105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 315/619] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:01.145956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.174982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d413c-59b5-452f-b827-6897678d40cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.200603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:01.214626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.215706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.216199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.216680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.227677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.229452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:01.247879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:01.249534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:01.250448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:01.250448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:01.250448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:01.250448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:01.252793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 305/620] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:01.261708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.308936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:01.354012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a6854-618e-4bca-8694-7e10485df681 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.390333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.391646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.391933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.392215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.414306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.415797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.421773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7a3c6-9efd-4b39-8680-778f308da01e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:01.450678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:01.452246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:01.453146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:01.453146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:01.453146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:01.453146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:01.453884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 316/621] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:01.458978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.505160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.505803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.506094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.506373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.507126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:01.507436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.508804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.528881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 306/622] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:01.534355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.565403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406795d-bf5b-45b7-81cd-87300ee9a1e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.589913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:01.602471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.603169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.603459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.603762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.604473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:01.604786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.606175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.615114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:01.619544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 317/623] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:01.625463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.743186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0144c-5fb4-41f5-b056-ec8a0a26dafa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.757931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b010-8df2-4a0a-b093-4c02c43c80b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:01.797738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.798523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.798828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.799108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.806364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.808071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.808185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:01.809208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:01.809696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:01.810182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:01.821493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:01.823240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:01.823383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:01.824260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:01.830071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 318/624] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:01.830678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:01.848297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:01.849262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:01.851236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 307/625] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:01.855784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:01.934352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.021054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.029140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cf3ab-b828-4391-9ff1-26c21d5b4223 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.048639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b666a89-d97e-40b9-9c68-7c8227a8c00a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.069644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.070743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.071126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.071468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.072351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:02.072697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.074266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.089388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 308/626] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:02.096129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.124849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.125639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.126265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.126614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.127872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:02.128262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.130393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2bf8f-9f83-4dff-aa11-a028aad79517 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.153916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 319/627] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:56:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:02.156312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ea64-b38f-44b0-b65b-1eb45c5c2900 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.156656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ea64-b38f-44b0-b65b-1eb45c5c2900 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.156997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ea64-b38f-44b0-b65b-1eb45c5c2900 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.162266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 320/628] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:02.162266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305ea64-b38f-44b0-b65b-1eb45c5c2900 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.215123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82b04-46ad-4cea-bc15-4e4c361b34fd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.272049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.273107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.273809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.274097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/eae0222c39364fa69be2a3cc938516bd/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.281113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305ea64-b38f-44b0-b65b-1eb45c5c2900 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.282943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.294201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=eae0222c39364fa69be2a3cc938516bd, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.296607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42291b-0074-46b7-adee-177af992c3e7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.315712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 309/629] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:02 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/eae0222c39364fa69be2a3cc938516bd/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:02.317454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.317958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.318473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.332930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.340806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:02.343495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:02.344296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.345819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.346185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.346466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.470652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 310/630] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:02.498218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.500121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.512275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.647732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086901b-ffb4-42ee-b304-18e67ec615ec None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.670643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 321/631] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:02.679797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.682210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.685321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.687076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.688372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.688869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.699109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.707619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:02.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:02.726229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:02.727157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 311/632] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:02.734172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.836166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895a6bf-bdfb-4379-b40f-fc7ebbfbb28d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.857325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51afcf-7c97-48d3-8e43-18f576aea215 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:02.872673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.874011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.874299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.874672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.885781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.887376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.893648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:02.894314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.894595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.894902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.895752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:02.896061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:02.897511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a191c4-10f5-4d95-b4bd-feb74d429cbe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:02.906367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 312/633] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:02.906515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:02.907420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:02.908356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:02.908356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:02.908356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:02.908356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:02.908589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a5d2-303c-47e2-9ebc-3a17eca786c9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:02.908589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a5d2-303c-47e2-9ebc-3a17eca786c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:02.908589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a5d2-303c-47e2-9ebc-3a17eca786c9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:02.913121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 313/634] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:02.913898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 322/635] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:02.915001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a5d2-303c-47e2-9ebc-3a17eca786c9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.917128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:02.976569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:02.980874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.025249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e40606-b6ef-4a9a-bd93-a98d426dc14b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.027740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a5d2-303c-47e2-9ebc-3a17eca786c9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.062048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.062845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.063127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.063405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.063977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.064637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.064923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.065202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.065931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:03.066499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.067638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996be30d-7ba1-4c54-a978-c0d96beaf9f3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.075316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.077001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.084061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 314/636] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:03.084061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.084061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.084061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0a123d65-7b2e-495b-81a0-ead125483c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:03.095245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:03.096157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:03.097063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:03.097063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:03.097063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:03.097063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:03.097063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 323/637] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:03.097773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd504be-1b6c-486c-bc43-360b0c7ba398 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.098056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd504be-1b6c-486c-bc43-360b0c7ba398 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.098401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd504be-1b6c-486c-bc43-360b0c7ba398 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.099833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 324/638] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:03.100538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd504be-1b6c-486c-bc43-360b0c7ba398 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.103708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:03.106120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:03.183357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 315/639] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:03.183357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.218435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd504be-1b6c-486c-bc43-360b0c7ba398 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.226072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.238410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.256310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.257098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.257744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.257744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.264480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.266214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-44f1404f-0b31-4c94-b20e-20aa19af777f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:03.279657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:03.280586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:03.281481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 325/640] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:03.290743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.291274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.291408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.297376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d0fe3-eb62-4a7e-8f53-b487c6f76cbf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.309435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:03.310641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:03.333057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.333795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.334147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.334495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.335261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:03.335860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.336958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.352183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 316/641] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:03.353201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.385160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 326/642] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:03.395714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.461953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929a0cc-243a-44e6-802a-d018e034f662 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.471366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.497474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.498376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.499253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.499598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.506830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.510806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:03.517376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510fd91-fdd2-4e7b-8450-218a313128c1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:03.519080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:03.520251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:03.521105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:03.523053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 317/643] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:03.523719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.557977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.558657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.558974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.559258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.560054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:03.560367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.562397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.577693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 327/644] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:03.578752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.634612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052275a5-b1ff-4e45-9207-a5099a38d241 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.667513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.668297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.668599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.668864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.676763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.678308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.688064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70923d68-6c58-413b-9a9b-efc9662a8908 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:03.693595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:03.694524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:03.695661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 318/645] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:03.696300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.726894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.727955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.728717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.729219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.729528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.742403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:03.752941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:03.753850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:03.754719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:03.759238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 328/646] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:03.760063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.780286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.800492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d9304-228b-4060-9c29-01fed2147a2e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.834659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.835346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.835626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.835899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.836633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:03.836936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.838440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.851223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 319/647] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:03.851223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.870750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3b63-ab25-4ae7-a2ef-ce3408e4d699 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:03.907030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:03.907681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:03.908075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:03.908350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:03.909288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=f1d06e6ad60240329d83bc4767439ee5) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:03.909600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:03.911179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:03.925433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 329/648] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/role_assignments?user.id=f1d06e6ad60240329d83bc4767439ee5&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:03.925433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:03.954764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:03.966942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01346ee1-0c80-4799-a606-d233f39b8179 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.013019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.013716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.014190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.014535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.015611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:04.015993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.017615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.034263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.039927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 320/649] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:04.041231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.059734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c211d96-b480-4da2-89b4-5152317493ff None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.111910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.112591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.112875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.113153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.113906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:04.114215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.115617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.130360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 330/650] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:04.130360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.168972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83291a9-1ad8-4ac6-87a0-641aaa0c4b48 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.211679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.213146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.213146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.213555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.228787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.230247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.240518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:04.250296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:04.251316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:04.252642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:04.252642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:04.252642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.252642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:04.252642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 321/651] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:04.252943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.270786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08185cd-d38f-4933-a38a-dfa03f051822 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.306462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.307142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.307406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.307961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.320909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.376560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a0048-2a69-462e-90ff-d17a38a70953 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.390953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.392594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.426541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.427733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.428468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.428970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.444124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.446721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:04.471705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:04.472690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:04.473613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:04.473613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:04.473613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.473613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:04.481167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 322/652] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:04.481167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.511858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 331/653] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:04.513089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.543301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.562476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.593152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c97b-6925-4753-993c-a0498eea8e7b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.622216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990bc41-328d-4a2e-a483-fd3ceffe14dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.627991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.628824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.629370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.629651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.630378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:04.630685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.632183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f63c9-5310-415e-9fd3-9c72c7490825 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.645014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 323/654] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:04.646345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e6465-4265-47cf-bd18-d81b50fb1418 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.646640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e6465-4265-47cf-bd18-d81b50fb1418 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.646954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e6465-4265-47cf-bd18-d81b50fb1418 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.648340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 324/655] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:04.648948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e6465-4265-47cf-bd18-d81b50fb1418 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.667948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b5406-9284-48b1-9805-a199685618aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.668553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b5406-9284-48b1-9805-a199685618aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.668837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b5406-9284-48b1-9805-a199685618aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.669118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b5406-9284-48b1-9805-a199685618aa None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.669829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b5406-9284-48b1-9805-a199685618aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:04.670140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b5406-9284-48b1-9805-a199685618aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.671556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b5406-9284-48b1-9805-a199685618aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.683823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 332/656] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:04.689353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b5406-9284-48b1-9805-a199685618aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.754964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e6465-4265-47cf-bd18-d81b50fb1418 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.768634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.792065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.792969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.793254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.793588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.795362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b5406-9284-48b1-9805-a199685618aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:04.796878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:04.806048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.808136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c3bb3565-0a5b-47d6-8b95-e05823c0dc2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:04.826983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:04.828090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:04.829145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:04.829145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:04.829145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:04.829145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:04.829839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:04.830630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.830953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.831221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.831646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 325/657] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:04.833049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:04.833337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:04.833613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:04.843284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:04.844721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:04.854287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:04.855528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:04.859688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:04.860588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 01 06:56:04.861526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:04.865465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 333/658] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:04.866068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.930789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 326/659] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:04.930789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:04.973984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c46962-f020-448f-a2bd-75c64b8da6b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.008884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.009665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.009948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.010223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.018425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.019966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.030343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:05.034387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:05.035530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:05.036525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 334/660] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:05.037129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.043042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565cbd2-d411-4352-a5b0-c36cd81df35c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.062828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.080384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.081044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.081328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.081604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.082315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:05.082619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.084027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.096234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 327/661] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:05.097269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.152809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cc71d-8bf1-49bd-9238-8e5eaaeda535 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.199190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a856b-e921-459f-bc66-1fe15316d5ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.200171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.201011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.202630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.203016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.203917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:05.204235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.205680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.224975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 335/662] 10.222.0.48 () {56 vars in 1139 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:05.225803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.243313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.244209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.244491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.244868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.256753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:05.264719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:05.265739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:05.266641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 328/663] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:05.268033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.283003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.313557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.334519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7edf00-aa33-4017-88c4-df29dbfedd3a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.371485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.371688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e07c-d6b5-4d3e-9f93-6efb06eb0758 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.372401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.372893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.373446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.375532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:05.376110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.378628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.400527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 336/664] 10.222.0.48 () {56 vars in 1137 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:05.400527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.409933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.410715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.411028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.411298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.418305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.419872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:05.435390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:05.436923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:05.437806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:05.441793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 329/665] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:05.442737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.496807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a542-a758-4a57-a205-853a16811be5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.518469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.532937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.533584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.533869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.534149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.542301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.545733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc22e-147a-4c7e-b4ce-b6b424dbe24d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.584544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.585161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.585446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.585927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.586458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:05.586783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.588151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.600913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 330/666] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:05.600913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.612496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.614315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.708656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e71e88-7d13-48f8-b70d-d34978776f5b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.714167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 337/667] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:05.715316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.749595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.750335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.750617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.750936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.755138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.758991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.821494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8bebd-c064-4bf5-9618-7d3c39c15e16 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:05.823898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.825354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.856396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:05.857056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:05.857338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:05.857615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:05.858999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:05.859309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:05.860731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:05.872415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 338/668] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:05.873224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.934253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 331/669] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:05.935352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:05.972769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.976193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:05.982032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9426cac-d379-4bdd-a5b0-da47d84c0d74 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.017574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.018366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.018641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.018947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.031213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.032651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.040726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9804c-e4cd-4be7-95d2-a241c75565b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:06.051769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.052656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.053539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:06.053539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:06.053539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.053539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:06.053539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 339/670] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.055137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.080874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.081669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.081948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.082220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.094047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.095533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:06.114948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.116023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 332/671] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.121549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.156329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adfb29-84fc-4489-9d4f-962c54266672 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.191591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.191591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.191591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.191591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.192056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:06.192056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.192561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.198379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 340/672] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:06.199185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.206157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.220828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.228883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5d17e-3ac9-4262-a819-fdd87ae0251f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.272781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.273593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.273879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.274153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.285845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.287358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-27a64187-6fd1-4c1e-8aa5-e81001b3ff00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:06.306936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.308213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.309146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:06.309146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:06.309146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.309146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:06.309146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 333/673] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.309874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c894-ca3b-45f7-ae70-3743b48a52b0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.310224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c894-ca3b-45f7-ae70-3743b48a52b0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.310404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c894-ca3b-45f7-ae70-3743b48a52b0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.312527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 334/674] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:06.313172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4c894-ca3b-45f7-ae70-3743b48a52b0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.321847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ff18-cfeb-49af-9bff-b46b1f8581ba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.366715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.367349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.367636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.367916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.368660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:06.368964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.370350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.393253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 341/675] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:06.394112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.436678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4c894-ca3b-45f7-ae70-3743b48a52b0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.471690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.472343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.472622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.472899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.473616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:06.473920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.475326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5fc23-bc47-4fd0-a8f7-66f2fb6b6f8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.482123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.484482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.489040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 335/676] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:06.490513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.495174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.495497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.510709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea16713-901a-4b61-8726-a56bef7d6c4b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.511699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:06.512902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:06.551642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.552303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.552687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.553095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.553814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:06.554121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.555650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.572870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 342/677] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:06.572870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.586814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 336/678] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:06.586814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.692868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ac0ab-be88-476c-8ee3-b3a9ff449145 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.711803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af1cc1-0ad7-409a-aee0-606bee6a9239 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.729608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.730690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.731002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.731437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.734191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.740705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.742563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.745086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:06.752174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.752972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.753253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.753523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:06.759401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.760355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:06.766875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:06.767441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.767523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 343/679] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.767523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.767647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:06.779648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.781082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:06.782007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 337/680] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.783169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a8931-2354-4a44-9b41-c58f954adc90 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.890916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e97a0-53f0-4245-a121-9582a4f8503c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:06.916355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.917142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.917423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.917896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.924473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.925484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:06.925914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.926268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:06.926543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:06.926838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:06.937761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:06.939224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:06.940136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.940972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:06.941956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 344/681] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.942446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:06.963199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:06.964095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 338/682] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:06.970600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.007886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.012940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.058377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6ac2d-7c6c-482a-9176-5aca71629ada None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.086817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c549-d9c6-4fc3-93ab-99656bc1cee9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.091083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.091743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.092022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.092855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.093694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.094004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.095457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.112003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 345/683] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.112794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.128925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.129925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.130414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.130962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.131949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.132439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.134141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.147549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 339/684] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.148523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.222131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2c998-4e84-41df-91d1-3fbf478cdf6f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.236124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.259251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cea22a-1265-407f-ba1d-e720fc5b7e55 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.260756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.261090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.262586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.262976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.263263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.264771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.265031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.266462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.284835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 346/685] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.284835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.301746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.302567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.302891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.303172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.304186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.304494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.305875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.318925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 340/686] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.319741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.397372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def577f-72cd-4488-833f-30d13b3b9650 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.430476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944b8c9-db45-423e-97be-94baaea7ebe2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.431940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.432561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.433112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.433388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.470415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.471098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.471386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.471667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.472546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=58947ba37ab14f61adde0bf2f34829f9) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.472811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.474246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.476889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.480444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 341/687] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/role_assignments?user.id=58947ba37ab14f61adde0bf2f34829f9&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.481246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.513956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.517458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.519255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.598106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6496db-77db-40d1-8336-61f631d929c9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.633619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 347/688] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:07.634683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.641614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.642287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.642572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.642881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.643730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=52d7bcc4b67c495f821519996cc2ab26) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:07.644052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.645489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.652407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 342/689] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/role_assignments?user.id=52d7bcc4b67c495f821519996cc2ab26&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:07.653227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:07.715817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.752775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb28990-77f0-43cc-af90-63b8b7a2416b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.774132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:07.792635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c211403-3523-4893-9271-ee4d33787b0d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:07.802522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.803676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.804171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.804630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.805540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.807364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.833832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 348/690] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:56:07 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:07.841228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:07.842187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:07.842476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:07.842756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:07.856572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:07.858701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:07.879213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:07.880217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:07.881557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:07.881557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:07.881557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:07.881557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:07.881557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 343/691] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:07.886556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.006212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a65f6-ec5b-448b-8dec-3ea5055816aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:08.035550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:08.043865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:08.044539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.044822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.045102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.046563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:08.046919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:08.048313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:08.062489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 349/692] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:08.068072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.177311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519db10e-cbe5-4da5-90e6-89186fd25245 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:08.215256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:08.216114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.216399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.216679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.224484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:08.225991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-04b61ce2-19d9-421e-9f47-889571f8bd1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:08.241278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:08.242200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:08.243149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 344/693] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:08.248573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.355109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7398a8-8882-48fd-8e6a-c8f0a62d238e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:08.366104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5dd5-21b8-453c-8426-a7fb502e6fc9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.366406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5dd5-21b8-453c-8426-a7fb502e6fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.366648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e5dd5-21b8-453c-8426-a7fb502e6fc9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.368051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 345/694] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:08.373036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.373379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.373664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.384095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:08.401388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:08.402929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:08.409790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:08.410750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.411297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.411971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.412957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:08.413475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:08.415199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:08.442203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 350/695] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:08.449316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.517950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 346/696] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 01 06:56:08.534596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:08.550680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.634898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04b471-733b-46ef-b44b-487667e8d49e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:08.676788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:08.677834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.678330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.678829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.679852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=eae0222c39364fa69be2a3cc938516bd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:08.680380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:08.682002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:08.689331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 351/697] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:08 2022] GET /identity/v3/role_assignments?user.id=eae0222c39364fa69be2a3cc938516bd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:08.751660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62313b-848f-42e3-9e80-5af1b98f244e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:08.801777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:08.807522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:08.808767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:08.809229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:08.809697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:08.831144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:08.838651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:08.839608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:08.840487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 347/698] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:08.845867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:08.988457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514419a-989d-4755-9ea5-025eb60516b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:09.002529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:09.024700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:09.025384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:09.025673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:09.025944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:09.026719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:09.027065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:09.028472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:09.037832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 352/699] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:09.042812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:09.151320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0f674-8d58-479f-9add-1478d6b384c5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:09.192542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:09.194085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:09.194554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:09.195034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:09.208073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:09.209873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:09.227639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:09.228557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:09.229456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:09.229456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:09.229456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:09.229456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:09.229456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 348/700] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:09.234792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:09.382014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6b484-e5e2-46b5-96fa-120559f10511 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:09.418418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:09.419124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:09.419409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:09.419688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:09.420989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:09.421302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:09.422719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:09.437365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 353/701] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:09.442461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:09.521185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:09.585218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199a2da-6da0-422e-8719-d2842ec12ecf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:09.631530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:09.632379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:09.632605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:09.632884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:09.645640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:09.647119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:09.667755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:09.668717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:09.669729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 349/702] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:09.674463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:09.738157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:09.818128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38403b-103e-4fb7-8b57-1516fa77bc17 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:09.852205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:09.852874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:09.853158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:09.853435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:09.854265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:09.854574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:09.856100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e3a5c-1709-46c6-8848-43f3d35ad88c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:09.874912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 354/703] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:09.879957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.008636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c58e-7426-40db-afea-f0d1ccc0d554 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.010902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.055041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.055695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.055977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.056250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.149472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.150951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.177655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff284c2-e496-4f6c-a603-f9b3972c9dca None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.178160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff284c2-e496-4f6c-a603-f9b3972c9dca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.178236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff284c2-e496-4f6c-a603-f9b3972c9dca None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.180409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 355/704] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:10.185849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.186147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.186422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.222791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:10.222791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:10.296368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 350/705] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:10.301638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.305127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.345513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 356/706] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:10.368308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.402808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.435055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434675-0906-440b-84ee-08e07d63c902 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.471713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.472532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.472814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.473091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.485794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.487265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.487559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa34b2f-45e7-4c91-a540-fb87dfa28e64 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-143783f8-c655-44f1-af31-19cefdd7a35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:10.505327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:10.506303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:10.507228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:10.507228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:10.507228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:10.507228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:10.507228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 351/707] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:10.509865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c586f-9404-4650-8aa6-c14af7118f98 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.512848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c586f-9404-4650-8aa6-c14af7118f98 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.513142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c586f-9404-4650-8aa6-c14af7118f98 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.515420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 352/708] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:10.516133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c586f-9404-4650-8aa6-c14af7118f98 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.527836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.528629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.528907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.529179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.530887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.532591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-18e3a5ec-754e-404d-a490-80c5e3e4cc37 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:10.549064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:10.550136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:10.551032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 357/709] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:10.551850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.551850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.551981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.579435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:10.580643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:10.593560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.643480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c586f-9404-4650-8aa6-c14af7118f98 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.662982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 358/710] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:10.662982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.684546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.686657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.687389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.687673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.688123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.689703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:10.690015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.691437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683ef2a-71c5-4831-864d-2e797906eb0e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.709169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 353/711] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:10.709169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4004c-6998-4242-9e02-a7c8efc4dbc7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.709169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4004c-6998-4242-9e02-a7c8efc4dbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.709169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4004c-6998-4242-9e02-a7c8efc4dbc7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.710897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 354/712] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:10.712127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4004c-6998-4242-9e02-a7c8efc4dbc7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.770405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e5f8-20a4-47b5-9b2c-dabb89712f3b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.806145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.807116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.807547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.808224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.809178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:10.809758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.811336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.814827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4004c-6998-4242-9e02-a7c8efc4dbc7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.820978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 359/713] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:10.822019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.832191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.851250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.852056 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.852536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.852647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.853265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.854626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cc3b0ac9-8f5b-4eda-aec1-17146460507b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:10.868435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:10.869489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:10.870331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 355/714] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:10.871232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.872046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.872046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.890404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:10.891606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:10.916251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:10.937867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60e266-1e40-45e6-a381-436d157e41af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:10.965089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 356/715] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:10.966029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:10.984498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:10.985600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:10.986187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:10.986633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:10.994374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:10.995908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:11.010903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:11.011837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:11.012733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 360/716] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:11.013214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.080599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5342e1-ba57-40ac-b5c7-1246af1cd335 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.092783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.117707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.118213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabcda-4b41-4218-9911-1a8a0d4710dc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.118395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.118657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.118967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.119684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.119998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.121372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.127642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 357/717] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.128425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.168701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.169988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.170977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.171691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.172692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.173213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.176097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.187378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 361/718] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.187378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.195408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.260616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c276517-e2ec-4201-82c9-eb28f2882df6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.297949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.298958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.299336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.299706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.313077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:11.321150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:11.322330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:11.323329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 358/719] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:11.324071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.335616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde36db6-6112-4dc1-872a-991205fa22dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.371179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.382240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.383375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.383717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.384129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.385063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.385377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.387681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.407125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 362/720] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.408127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.449562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37205-c375-4499-8f01-de2aa2414528 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.477443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.485989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.486663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.486988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.487274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.487992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.488306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.489684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.498918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 359/721] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.498918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.523942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea32e9-96da-4b48-88d7-29978f987f0e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.562487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.563323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.563632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.563872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.564630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.564937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.566323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe177d-9fdd-44df-8965-ad67b2fe72b0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.575609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 363/722] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.576984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d98435-4d24-4a03-81c9-cf3098328dbb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.577270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d98435-4d24-4a03-81c9-cf3098328dbb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.577545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d98435-4d24-4a03-81c9-cf3098328dbb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.579038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 364/723] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:11.581970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d98435-4d24-4a03-81c9-cf3098328dbb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.657136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.663272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12a62-6f38-4285-a181-c7034e8f1201 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.702639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d98435-4d24-4a03-81c9-cf3098328dbb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:11.714020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.714685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.714997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.715279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.716113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:11.716421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.718382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474c485-0abb-4b18-9a05-f0ea1e588ae7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.724520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 360/724] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:11.725891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.726181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.726462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.736034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:11.745376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:11.746621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:11.747765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:11.749298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:11.750901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:11.751469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:11.767459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:11.768946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:11.797174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:11.798144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:11.799066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:11.799066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:11.799066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:11.799066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:11.802436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 365/725] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:11.810101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.868286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 361/726] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:11.918968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:11.984579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7526241-f6ca-48bc-81dc-ffd0899e4785 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.023395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.024065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.024348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.024621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.025347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:12.025652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.027044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.036157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:12.051415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 366/727] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:12.052543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb339a49-9afb-4d2f-a9f7-9b11d14d5935 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.056541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.115579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:12.126301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.127130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.127413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.127685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.159759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:12.167424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:12.168345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:12.169230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:12.174788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 362/728] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:12.180442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5cb71-d500-4995-b731-17b006510b08 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.187786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.222002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.222908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.224023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.224312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.241117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.242588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:12.274347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:12.276585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:12.279031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 367/729] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:12.282496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.392085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f925bc-41a7-447b-b6b0-0209dd5e56b5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.407150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:12.411826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebc446-575d-4c7b-8c1e-2ab57fb13004 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.461733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.462326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.462611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.462917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.463675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:12.463989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.465368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.476339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.477008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.477289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.477570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.478415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:12.478726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.480168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.485064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 363/730] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:12.490400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.507055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 368/731] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:12.515811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.528162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:12.668960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42635a2e-1ec5-4411-b607-1ccfb4cd72d6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.710572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.712742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.713174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.714518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.718445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd6029-412f-48bc-8041-45fe350af97b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:12.753178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:12.802813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.804573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.816379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:12.817176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:12.817458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:12.817732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:12.841147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:12.842648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:12.880669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:12.881831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:12.882978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:12.882978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:12.882978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:12.882978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:12.891257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 364/732] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:12.896648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:12.913441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 369/733] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:12.918683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.014159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.031141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.084697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187406a-1b1d-4c66-8345-f9bd2778d63f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.102553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e3c85-f694-41d9-981e-aea4108c0f8b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.133168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.133964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.134246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.134526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.152083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.153521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.171719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.172387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.172670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.172947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.173677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:13.173987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.175394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:13.192064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:13.192962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:13.193851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:13.193851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:13.193851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:13.193851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:13.193851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 370/734] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:13.198783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.209736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 365/735] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:13.216156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.380774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d1b9-5713-4036-b755-d34f15ee3b05 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.395842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262320e-118b-4f74-a031-525b2f08e72e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.403281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.436392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.446006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.446711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.447065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.447850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.448593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:13.448909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.450300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb63d35-7fde-4880-91a8-c24f80dbf4f6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.455901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.456703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.456988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.457267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.469336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.471156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.471460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 371/736] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:13.472790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1913-85ec-4943-abca-6dcd807a5781 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.473080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1913-85ec-4943-abca-6dcd807a5781 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.473362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e1913-85ec-4943-abca-6dcd807a5781 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.478710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 372/737] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:13.480629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1913-85ec-4943-abca-6dcd807a5781 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ab568fe8-06b4-4eec-aaa6-9ac8ca9af0b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:13.496473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:13.497454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:13.498441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:13.505514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 366/738] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:13.507207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.507720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.508203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.530428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:13.532098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:13.593445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e1913-85ec-4943-abca-6dcd807a5781 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.622856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 367/739] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:13.624139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.642979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.644112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.645156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.646664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.658715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.661351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:13.681564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:13.682507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:13.683728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:13.689597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 373/740] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:13.689597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.729449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.750865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f613aa-7058-4fa3-a8de-28896d308086 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.773525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:13.799405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.801068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.801592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.802090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.803171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:13.804237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.806433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.816587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7d63f-af8f-4537-91a5-d48e6b9ea8bd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.831421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 368/741] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:13.832537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.857894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:13.858610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:13.858929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:13.859208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:13.859933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:13.861412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:13.882819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 374/742] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:13 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:13.882819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:13.971758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2570e0-4fc2-45ac-b2c4-d8cec05abd73 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:13.999724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.026824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db1da-c021-45e0-83e9-853edf2f5e35 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.031741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.032508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.032952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.033326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.034225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:14.034638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.036209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69447897-5eb3-4e23-b76e-08a8fd01cde1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.049444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 369/743] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:14.051051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42049a-1ec9-4d07-a257-6bc471a96c33 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.051342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42049a-1ec9-4d07-a257-6bc471a96c33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.051619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42049a-1ec9-4d07-a257-6bc471a96c33 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.052972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 370/744] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:14.053972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f24e-ae06-467c-b430-e92ab7ba04db None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.054262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f24e-ae06-467c-b430-e92ab7ba04db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.054630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68f24e-ae06-467c-b430-e92ab7ba04db None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.056024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 371/745] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:14.056883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68f24e-ae06-467c-b430-e92ab7ba04db None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.068113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.068771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.069052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.069333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.089318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.139993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.141457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7cd647-2227-4c31-ba7f-fbfd94e12fdd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.171355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68f24e-ae06-467c-b430-e92ab7ba04db None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.209249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.210026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.210190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.210442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.214152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:14.214152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.214152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b2cf3-5a96-4ac1-b0bc-227cd189a66a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.219251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 372/746] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/role_assignments?user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:14.224413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.225709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.226216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.237035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.252799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:14.255018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:14.261259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 375/747] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:13 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 01 06:56:14.261259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.261259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.261259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.279584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:14.281062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:14.359337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 376/748] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:14.360413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.362401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 373/749] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 01 06:56:14.382155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.418520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.499332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d33bb-5b98-4f14-a78c-fca685666184 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.515824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.517280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6092a-3c9b-443e-803f-12a40ebaaf9d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.538314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.539706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.540142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.540568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.551413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.553091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.561293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.562094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.562377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.562651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:14.572385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:14.573487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:14.574593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:14.574593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:14.574593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:14.574593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:14.574852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:14.577216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 377/750] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:14.578200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:14.583575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:14.584514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:14.585363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:14.585999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 374/751] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:14.586756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.696986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.708366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e533a5d-89d4-4438-a8a7-1919c7aa0953 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.715698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904874f3-5cf3-4f4b-89e6-9505fb3d7b32 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.751788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.752328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.752612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.752884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.753605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:14.753911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.755322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.758248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.759218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.759711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.760199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.761119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.763378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.768599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 375/752] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:14.769536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.787315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 378/753] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:14 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:14.788357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:14.823178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:14.919389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14c271-8938-490d-b17d-055184d174c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:14.962576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:14.963426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:14.963710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:14.963986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:14.972790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:14.974258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:14.991667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:14.992619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:14.993514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:14.998008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 379/754] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:15.000018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.036838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4956358-49c0-4513-ab2f-ca7c8fca13ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.096508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.097194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.097472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.097746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.098503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:15.098890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.100603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.108560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:15.112950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 376/755] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:15.118038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.128381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:15.130529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25116cb5-deca-4c1e-9b38-46579fc695af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.165398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.166312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.166747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.167202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.168159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:15.168618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.170721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.184626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 380/756] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:15.195028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.237504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cada7-d7fb-44d9-92c9-7a2f0547b54b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.273311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.274434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.274895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.275527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.288087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.289946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.297771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2496d94-89e3-4c83-97a5-051f5aa3b058 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:15.308847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:15.309906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:15.311093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:15.311093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:15.311093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:15.311093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:15.313091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 377/757] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:15.318098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.355947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.356724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.357105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.357519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.358494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=eae0222c39364fa69be2a3cc938516bd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:15.363000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.364674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.376477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 381/758] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/role_assignments?user.id=eae0222c39364fa69be2a3cc938516bd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:15.386506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:15.435595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8d6c2-ad95-4231-899f-d10d89a72c8f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.477477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.478560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.479103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.479609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.480582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:15.481131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.482848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.498943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 378/759] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:15.506893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.523874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:15.668299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb06af-5909-4a88-8aff-afee5b3272aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.703562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.704355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.704642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.704918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.717391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.718860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:15.740177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:15.741139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:15.743393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 382/760] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:15.752233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:15.888348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68f760-5840-47e2-aaed-edcd8c41e003 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:15.916970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:15.942146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:15.942746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:15.943120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:15.943398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:15.944246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:15.944556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:15.946055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3861-408a-44f2-9282-0da5fea70178 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:15.968101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 379/761] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:15.973258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.078124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:16.105200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5364f4-5df0-4218-aaad-58aad4bf5c58 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.105515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5364f4-5df0-4218-aaad-58aad4bf5c58 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.105745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5364f4-5df0-4218-aaad-58aad4bf5c58 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.107261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 380/762] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:16.116947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.117238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.117509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.138617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39967b3b-40be-400b-b8f7-1fcf51c8b18f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.141406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:16.143535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:16.203155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.204306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.204944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.205968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.243635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.245440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.249325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 381/763] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 01 06:56:16.271390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:16.272281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:16.273173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:16.274056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:16.274056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:16.274056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:16.274056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:16.276236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 383/764] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:16.281840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.457939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0fb25-5338-4331-bf4a-801138c31c35 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.474603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:16.494537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d890b-d5c0-4678-9bd8-4d6f3f2e240c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.520669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:16.540587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.541391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.541675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.541952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.542608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.544005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.555528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.556193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.556480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.556758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.557477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:16.558044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:16.567717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:16.568700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:16.569553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:16.572020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.577385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 382/765] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/services/network => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:16.581803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.584746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 384/766] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:16.594977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.712795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f35cfc-a074-4a09-aa6f-d0a10b5eafa1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.746648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1b9ee-c5c7-40ca-b4ca-340cc46ca931 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.752756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.753971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.754476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.755024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.763261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.765805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e48ed619-92ca-404f-a127-ae20c0da1008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:16.783941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:16.784919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:16.785836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:16.788043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.788796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.789083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.789471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 385/767] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:16.790937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbfb57-da90-47b4-b609-d9bf7a1500a8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.791391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbfb57-da90-47b4-b609-d9bf7a1500a8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.791835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbfb57-da90-47b4-b609-d9bf7a1500a8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.793493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 386/768] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:16.793987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.794752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbfb57-da90-47b4-b609-d9bf7a1500a8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.794918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:16.795216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.796601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1e4c6-026a-479e-b0bc-64796a399fd0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.807064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 383/769] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:16.809017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.809246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.809628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.834051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:16.835831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:16.840469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:16.867314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:16.904919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbfb57-da90-47b4-b609-d9bf7a1500a8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:16.919183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 384/770] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:16.920447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:16.940888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:16.941688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:16.946263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:16.946263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:16.946263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:16.946263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:16.946263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:16.957726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 387/771] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:16.965749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.048408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f5ea1-bda4-4013-9704-3d272dc7e453 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.068044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.072324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcfed2-ae2a-4f31-be10-830ee909ecba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.089669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.090882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.091335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.091755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.092636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:17.093109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.094753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.105847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 385/772] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:17.107227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.112449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.113144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.113553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.113939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.116270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=458dc4b52e3d4a13a8a4f801ddf2e6c4, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.117824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.135440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 388/773] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:16 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:17.143757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.144010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.229359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a05464-2f3c-4a83-90f5-adb71e03599d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.286946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.287928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef803064-44c3-42f5-8b6f-811c276ee4ce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.288401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.288886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.289647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.321400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:17.338669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:17.339818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:17.340730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:17.344907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 386/774] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:17.351212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.375572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.378199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.379360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.379961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.383310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] RBAC: Adding query filter params (interface=public, service_id=9b250aeacdb34d63afbe220bd868984d, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:17.383882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.386505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.399538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 389/775] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/endpoints?service_id=9b250aeacdb34d63afbe220bd868984d&interface=public®ion_id=RegionOne => generated 194 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:17.405747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.438441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.479197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.491638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80509b0-6d90-4a00-a479-d8717b3d2fe7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.527137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.527516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7bd09-4291-4207-8abf-ccfa59702a4c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.527976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.528070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.528337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.529088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:17.529417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.530795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b0933-c5c3-457a-973e-84e89183b184 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.539548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 387/776] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:17.540910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629146b1-88df-4520-b6d0-0f278e35b0ca None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.541204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629146b1-88df-4520-b6d0-0f278e35b0ca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.541482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629146b1-88df-4520-b6d0-0f278e35b0ca None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.547273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 388/777] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:17.548098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629146b1-88df-4520-b6d0-0f278e35b0ca None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.567193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.568267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.568788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.569208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.570028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.571722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010492c-9fc0-4377-9522-69c2d6a66169 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.578219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 390/778] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:17.584101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fbd98-b067-46dc-8ea2-12c35cec75cd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.584586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fbd98-b067-46dc-8ea2-12c35cec75cd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.585134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fbd98-b067-46dc-8ea2-12c35cec75cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.620035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-899fbd98-b067-46dc-8ea2-12c35cec75cd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:17.621721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-899fbd98-b067-46dc-8ea2-12c35cec75cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:17.657864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629146b1-88df-4520-b6d0-0f278e35b0ca None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.732061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 391/779] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:17.734694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504406-a5ac-4346-9bbc-dc4c7ecba0f5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.735144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504406-a5ac-4346-9bbc-dc4c7ecba0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.735672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504406-a5ac-4346-9bbc-dc4c7ecba0f5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.736537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.737187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 392/780] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:17.741434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.741434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.741434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.743308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.743593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.743866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.754045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.756008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.761900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:17.763139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:17.777428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:17.779043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:17.780534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:17.781436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:17.781436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:17.781436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:17.781436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:17.783411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 389/781] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:17.786928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.794237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:17.845884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 393/782] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:17.846591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.922473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23f69-be5c-4724-89d8-3d428197d17e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.958260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f655f9-104c-4494-9133-07fc289bc561 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:17.960242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.961388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.961938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.962421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.963333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.964929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:17.978758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:17.979758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:17.980632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:17.985421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 390/783] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:17.986601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:17.994934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:17.995638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:17.995917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:17.996234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:17.997188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:17.997491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:17.999084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.012363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 394/784] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.013131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.023663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.061758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.101227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52719b97-2511-489f-bd55-439804faa7b8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.123530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f713a-596e-4620-8bbc-75d7b3f0ff81 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.136948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.138113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.138606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.139657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.151026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:18.158896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.159550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.159828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.160158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.160860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:18.161187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:18.162101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:18.162967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:18.163612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.163612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.170860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 395/786] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.170860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.171292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 391/786] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:18.173341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.245667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.281465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4430-d0c2-4856-86df-4cc4a2d81105 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.317855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.318638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.318950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.319225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.327973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.329501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:18.345399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:18.346367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 396/787] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:18.348374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.426293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a9e9-7217-4498-8fcf-f0c33f48385d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.463220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.465647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.466690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.467267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.467754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.468443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a41b9-29c7-4bab-abaf-5ee7e3f0b1d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.468983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:18.469503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.472417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.479121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 392/788] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.480752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.508559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.517060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.517932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.518291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.518634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.519776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:18.520158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.521846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.536665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 397/789] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.536665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.597254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3157314-1a8c-4893-91df-0313beb703fc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.634819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.636497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.636993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.637477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.638513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:18.640014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.641522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.656008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa4696-d870-4bd6-b57f-c21409151d69 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.666005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 393/790] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.666973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.691426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.692317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.692773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.693218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.694188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] RBAC: Adding query filter params (interface=public, service_id=6d1b5ffc3f0a4fe5bae71590926971b3, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:18.694671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.696432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.702504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 398/791] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/endpoints?service_id=6d1b5ffc3f0a4fe5bae71590926971b3&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:18.703590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.728254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.763396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:18.794750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f91e-5af9-4ba2-837c-cb3b2b68f9ff None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.834977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.835806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.836079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.836357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.848844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.850370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:18.869884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:18.870899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:18.871799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:18.871799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:18.871799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:18.871799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:18.874961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 394/792] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:18.876038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:18.938130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aacdd-cead-455c-800f-bb6b527f36b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:18.977512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:18.978337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:18.978615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:18.978912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:18.989400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:18.990864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:18.995790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.000014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95303-d65d-469c-ad8e-1de15c1b9f44 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:19.008498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:19.009434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:19.010337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:19.010337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:19.010337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:19.010337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:19.010337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 399/793] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:19.011067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.041536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.042514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.043080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.043572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.044498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.046096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.052154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 395/794] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:18 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:19.053254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.135220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ef0a4-d084-4acb-9a56-f91a5d2831cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.146283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.175015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.176140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.176635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.177114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.178070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:19.178591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.180235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.198214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 400/795] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:19.205589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.222525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c689f-4161-4d7b-968f-5b2128af5233 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.258934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.259872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.260157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.260473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.261233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:19.261550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.263346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.279344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 396/796] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:19.284670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.317157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.320707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae214bb-d89a-4303-a4d1-7d6f6c74c435 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.360993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.361934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.362334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.362707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.370854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.378793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2e7a7e22-5c6d-4f2e-87b4-c8b24ff7453e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:19.388860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:19.390345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:19.391336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 401/797] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:19.392455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0433ee-f693-4784-9bc3-032fd0769eb9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.392455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0433ee-f693-4784-9bc3-032fd0769eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.392758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0433ee-f693-4784-9bc3-032fd0769eb9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.394174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 402/798] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:19.398963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0433ee-f693-4784-9bc3-032fd0769eb9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.422453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516097b-2e48-4d76-b1c5-5e2774229045 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.435441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.476687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.478208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.479414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.479947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.489709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.492815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-95b19aa1-2253-4b3a-a074-e8f8c657f542 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:19.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:19.511250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:19.512264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:19.513585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 397/799] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:19.515079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.515467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.515921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.524449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0433ee-f693-4784-9bc3-032fd0769eb9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.533131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:19.534446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:19.570437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.572791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.573233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.573656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.574663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:19.575139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.576859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.601550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 403/800] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:19.602651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.618797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 398/801] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:19.618797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.641753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.731089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.744371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe50c5-b974-4400-8e0f-5e54fdec6253 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.747586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6dfe-536f-422c-8d1e-8a6bed42db74 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.782132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.783059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.783282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.783555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.791602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.792978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.793690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.794150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.794263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.795767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:19.796278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:19.796384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.798229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:19.813466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 404/802] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:19.814650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:19.815272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:19.816139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:19.817163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:19.817163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:19.817163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:19.817163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:19.817163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 399/803] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:19.818749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.912555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:19.930368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8c1a3-35b7-4e0c-a6e5-8dba0ff44033 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.955036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890c661-44d8-44c5-b0eb-1490db60535a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:19.973145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.973937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.974341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.974625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:19.985339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:19.992755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:19.993883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 405/804] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:19.994740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:19.997789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:19.998815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:19.999127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:19.999475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/b73d15dbbe6d4bfc99ea858b7e12c3f8/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.008045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.014867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=b73d15dbbe6d4bfc99ea858b7e12c3f8, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.016708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c5c02-2d74-48af-b498-06c2eebed144 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.031153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 400/805] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:19 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/b73d15dbbe6d4bfc99ea858b7e12c3f8/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:20.032669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd553d-95da-4ffa-856a-a93b26e2b9f8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.033023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd553d-95da-4ffa-856a-a93b26e2b9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.033387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd553d-95da-4ffa-856a-a93b26e2b9f8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.039039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 401/806] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:20.039039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd553d-95da-4ffa-856a-a93b26e2b9f8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.136079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f3cb-852f-4bdb-91af-8417d9b619b0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.174707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.175791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.176268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.176741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.177674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:20.178302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.179795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a8e66-fd38-42d8-83e9-010990f1fd1b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.180272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd553d-95da-4ffa-856a-a93b26e2b9f8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.200111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.200304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 406/807] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:20.202493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.202805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.203135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.223941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:20.224717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.225112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:20.225453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.225741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.226077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.226862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:20.227390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.229616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.240649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 402/808] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:20.242192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.292681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.301612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 407/809] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:20.302616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.361621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe951e-b750-45c8-ab05-b1143b4a980a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.399040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.399831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.400113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.400386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.408015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04510-724f-4ac5-812b-2669169746f1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.408370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.409977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:20.425619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:20.427762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:20.429859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:20.434799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 403/810] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:20.434799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.442649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.443471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.443749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.444021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.454721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.456231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:20.473844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:20.476039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 408/811] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:20.477377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.477796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.526841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.547053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038c8ca-2d76-4833-86d3-02a2163b5750 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.582308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548db67-323e-45f0-99de-267ca5eada4b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.584719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.585513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.585791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.586063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.586719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.588092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:20.601209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:20.602210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:20.603446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 404/812] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:20.604257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.626602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.627341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.627706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.628004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.628793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:20.629107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.630645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.647302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 409/813] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:20.648230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.728515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6d54-8397-49c0-bcb4-f756ab1341e7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.730087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.767008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.767674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.767985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.768283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.768965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:20.769274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.770636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.783111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 405/814] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:20.784033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.789442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e6c3-f697-41c2-a14c-77041432693f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.802753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:20.841623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.842872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.843886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.844364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.845584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:20.846334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.848615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.855753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 410/815] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:20.857004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.897400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7d134-7c4a-481a-8231-e2d0e65cc429 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:20.943902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:20.945341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:20.945824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:20.946308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/eae0222c39364fa69be2a3cc938516bd/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:20.949428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=eae0222c39364fa69be2a3cc938516bd, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:20.952404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:20.979172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 406/816] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:20 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/eae0222c39364fa69be2a3cc938516bd/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:20.979172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:20.986150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4e588-c87d-4d97-b0fc-341b057e13ca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.029049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.063860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.064714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.065004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.065282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.081790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.083303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:21.101433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:21.102467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:21.103475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 411/817] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:21.109523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.184403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.193178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3009b-4506-4182-a2ad-ca19a5c84d41 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.239102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.239787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.240143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.240482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.241358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:21.242449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.243349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.249546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 407/818] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:21.255121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.280075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786507ad-2200-4ff3-9694-9d8682155231 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.334843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.335684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.335975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.336579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.337711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:21.338028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.340251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae674fa-cd19-4bde-811a-ea22d8d39cac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.362270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 412/819] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:21.363617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d1cc0-7e9b-4079-9419-8e8f91835878 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.363912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d1cc0-7e9b-4079-9419-8e8f91835878 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.364189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d1cc0-7e9b-4079-9419-8e8f91835878 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.365694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 413/820] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:21.371005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.371357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d1cc0-7e9b-4079-9419-8e8f91835878 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.385157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077a14-9033-4c8a-8c20-e414bad63e94 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.419305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.419987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.420827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.421107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.421849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.423244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8222199-4016-4bf8-ac26-26313fc161c2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.450120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 408/821] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:21.451677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.452022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.452307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.471173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:21.472400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:21.515529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.551775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d1cc0-7e9b-4079-9419-8e8f91835878 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.601286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 409/822] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:21.613170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.613936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.614220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.614511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.638821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.711428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.713035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.741989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.774349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e23f3-ba34-412e-8d63-a49aa1f6e51c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.809158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.809968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.814158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.814158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.814158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.814158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.814158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:21.828953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:21.830150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:21.835829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 410/823] 10.222.0.48 () {56 vars in 1129 bytes} [Fri Jul 1 06:56:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:21.839036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.850545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 414/824] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:21.852277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:21.945946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed148ef9-332a-4ec6-97c7-4fcf69f7db29 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.968924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8278b-2b2b-4210-9a26-fb28e1064210 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:21.986200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:21.986200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:21.986200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:21.986200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:21.986625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:21.986625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:21.987392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:21.991248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:21.996318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 411/825] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:22.001934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.019344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.020580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.021072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.021570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.033767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.035519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:22.054688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:22.055652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:22.056531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:22.056531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:22.056531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:22.056531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:22.064386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 415/826] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:22.065899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.111002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393c372-f47c-413e-92e9-3540f2ff15ba None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.114199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.152584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.153255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.156343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.156343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.156343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:22.156343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.156343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.161675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 412/827] 10.222.0.48 () {56 vars in 1124 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:22.167191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.193751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bbafd6-cbfa-4ba5-85bf-a7591a91631b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.233908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.234576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.234886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.235165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.235889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:22.236190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.237578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45bcae-8cea-4b11-a122-87a691dec6e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.238651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.250816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 416/828] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:22.251292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5a5d-3a24-4875-8da5-012b90aebe97 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.251528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5a5d-3a24-4875-8da5-012b90aebe97 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.251801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5a5d-3a24-4875-8da5-012b90aebe97 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.253137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 417/829] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:22.258067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.258349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.258623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.275411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:22.276765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:22.285514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1891205-4334-4460-b408-8ba3e20d41a3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.320915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.321569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.321848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.322127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.322878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.324230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.350799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 413/830] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:22.350799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.364594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 418/831] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:22.388417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.423741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.476137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd1b9f-6987-4078-832c-7b7f9c0e4519 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.501039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.521613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.523108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.523351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.523688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.533289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.534726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.536244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b745508-47f6-4537-80c6-056a1db58f3e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:22.551346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:22.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:22.553218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:22.557426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 414/832] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:22.558513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.578946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.580134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.580711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.581184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.592565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:22.600889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:22.601839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:56:22.602697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:22.605480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 419/833] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:22.613029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.684474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a64050-c5c9-4cab-8558-6f3067f1593e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.706028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.727236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4bda4-52e9-4e8b-bc3a-a90843f0fe3e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.733174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.733843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.734129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.734409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.735250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:22.735563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.737644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.755394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 415/834] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:22.760710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.770045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.771067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.771553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.772044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.772989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:22.773493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.775232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.784283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 420/835] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:22.789903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:22.816329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:22.879809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146b1d0-d515-4512-8a34-3dce0febc2a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.922266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041a5f-f583-44d1-978d-261201295f72 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:22.924930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.926462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.926747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.927052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.927883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:22.928194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.929613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.939127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 416/836] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:22.968681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:22.969969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:22.970466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:22.970981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:22.984238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:22.986054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:22.994317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:23.006752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:23.007789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:23.008696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:23.008696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:23.008696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:23.008696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:23.011019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 421/837] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:23.019997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.133804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab37c7b-875d-4cf6-8e33-ddd6db7e5f74 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.136356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.169256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.169916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.170226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.170496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.171247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:23.171597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:23.172958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240c01a-07cb-48b8-81fc-cf5d82c1e566 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:23.186621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 417/838] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:23.194411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.307306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cb51c-a007-4c1c-bed1-1045965f91bf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.331328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e030-3bec-4aa0-9c9a-1a62b4f5926c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.331651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e030-3bec-4aa0-9c9a-1a62b4f5926c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.331877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e030-3bec-4aa0-9c9a-1a62b4f5926c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.333463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 418/839] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:23.338353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.338643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.338945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.344676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.345735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.346215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.346703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.355636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:23.355804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:23.356991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:23.358216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:23.373157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:23.374662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:23.375758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:23.378359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 422/840] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:23.383923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.435020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 419/841] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:23.445695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.456192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.490472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ea613-85a6-4531-906b-b194bd6ce153 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.525363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.526406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.526929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.527206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.528047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:23.528350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:23.529816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:23.548171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 423/842] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:23.553256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.562021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.571270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed837-e50d-4b66-ad01-566aa442da97 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.617137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.618593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.619057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.619762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.635312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:23.643337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:23.644463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:23.645461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:23.647948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 420/843] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:23.653265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.694333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41b000-fb51-4fe1-88f4-7459032bc179 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.711857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.731536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.732349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.733203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.733550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.788596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81ea8b-97c2-4f1b-83b1-039ff7faf26a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.820717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:23.822730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:23.826508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:23.827446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:23.827934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:23.828420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:23.829364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:23.829872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:23.832123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:23.848833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 421/844] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:23.849472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.872106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:23.965809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e4b02-5a74-44cf-9129-20281ca38244 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:23.966646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 424/845] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:23.968253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:23.986170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.002157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.002990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.003328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.003600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.015577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.017026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:24.036355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:24.037299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:24.038205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:24.038205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:24.038205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:24.038205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:24.042224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 422/846] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:24.043212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.077660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bef856-599f-438f-a2ad-820344aaf723 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.113198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.114374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.114887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.115367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.115473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.126436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.128228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:24.146928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:24.147856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:24.148762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:24.148762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:24.148762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:24.148762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:24.154976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde805-c80a-4b71-b651-aa457040cbb4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.155782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 425/847] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:24.156735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.196599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.197609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.198083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.198500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.199469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:24.199916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.201976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.222564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 423/848] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:24.224289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.235263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.270288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7e6b2-40bb-413a-9ba4-59dee34f5cbf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.305294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.306004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.306274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.306553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.307322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:24.307633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.309271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.325237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 426/849] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:24.326129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.337676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b4a5f-7265-4082-9d2b-c4f5e05beb22 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.367377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.378632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.379634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.380085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.380503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.388510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.390995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:24.406253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:24.407505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 01 06:56:24.408635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:24.415026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 424/850] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:24.416483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.429004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce872294-56e6-4516-b3ed-a75426e00c48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.456766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.464817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020a15-943c-4102-9320-a06181620024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.465417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020a15-943c-4102-9320-a06181620024 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.465706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020a15-943c-4102-9320-a06181620024 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.465990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020a15-943c-4102-9320-a06181620024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.473339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020a15-943c-4102-9320-a06181620024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.474727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020a15-943c-4102-9320-a06181620024 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-23020a15-943c-4102-9320-a06181620024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:24.488903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:24.489957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:24.491003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:24.495298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 427/851] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:24.495882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020a15-943c-4102-9320-a06181620024 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.532431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5a1f5-72f9-486c-be9c-e85cda39b35d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.569024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.569688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.569967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.570802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.571167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:24.571406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.572818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f8a6d-40fd-4bc7-8649-a49ba01ebbe7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.591020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 425/852] 10.222.0.48 () {56 vars in 1135 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:24.596467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b80b2-d57c-4839-a676-c673dcbc30b7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.597061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b80b2-d57c-4839-a676-c673dcbc30b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.597346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b80b2-d57c-4839-a676-c673dcbc30b7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.599037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 426/853] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:24.599778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b80b2-d57c-4839-a676-c673dcbc30b7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.606241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020a15-943c-4102-9320-a06181620024 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.646696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.647375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.647686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.647926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.648705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:24.649014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.649401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.650552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca118a2-4e62-494d-bf14-e9d70a1c734a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.663241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 428/854] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:24.664450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.664735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.665008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.684262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:24.685104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:24.712031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b80b2-d57c-4839-a676-c673dcbc30b7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.731356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.747361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.748005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.748856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.749129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.766834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 429/855] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:24.766834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.824254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.825938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.874899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:24.879315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b0b78-231d-440e-a72b-8d27e0418055 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:24.914112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:24.914761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:24.915094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:24.915373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:24.916356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:24.916496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:24.917907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:24.924410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 430/856] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/role_assignments?user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:24.925001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.957134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 427/857] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:24.962698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:24.988582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.075947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c11c3-9d88-49aa-bdb2-10653c5d74dc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.086437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1572f90-5f67-4402-9dd8-2308c642da4b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.119991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.120909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.121193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.121456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.123146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.125032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.130107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.130107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.130696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.131002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b44d4e08-0268-4285-9b36-37c194e77c32 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:25.137689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:25.138729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.139806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 431/858] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:25.140700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442597f-741b-480f-b23f-9cd12e7720b4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.140700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442597f-741b-480f-b23f-9cd12e7720b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.141003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442597f-741b-480f-b23f-9cd12e7720b4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.142393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 432/859] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:25.144912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.146460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.146820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442597f-741b-480f-b23f-9cd12e7720b4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.155283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-035fd9a5-4b74-44b8-863c-f08b5c839c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:25.167676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:25.168599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 428/860] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.171549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.192327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:25.192327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:25.267000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 429/861] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:25.267792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.271693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442597f-741b-480f-b23f-9cd12e7720b4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.280447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.311143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.312036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.312461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.312894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.313745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.314201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.315733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.320363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 433/862] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:25.321356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.381888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd815-34bb-410d-817b-6903bd805f8f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.408996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.418671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.419540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.420547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.420995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.421723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.422155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.423413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.430344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c76e99-4603-483d-8471-c14d63a42484 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.440897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 430/863] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:25.440897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.471510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.472318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.472597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.472878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.473538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.475016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:25.488489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:25.489532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.495366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 434/864] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:25.496224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.531384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.554057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0a3a-b564-4de3-89df-dce08dd88729 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.589137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.589792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.590072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.590346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.591084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.591391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.593343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.597942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 431/865] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:25.603798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cfd02-c0d1-4ddf-8d25-c576eb0417e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.604021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.644434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.645652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.646143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.646624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.655578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.657488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:25.675090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:25.676071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:25.677000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:25.677435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.683800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 435/866] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:25.687379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.745458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8895e07-1066-41e3-b7f3-535e14c8e34b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.793684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.794268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.794546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.794847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.795579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.795884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.797236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.802191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 432/867] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:25.807872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.810593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.819882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aac00b-f23a-4c15-90b1-1fd242312202 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.863680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.864996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.865434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.866257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.867512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.867980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.869803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:25.886559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 436/868] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:25.892302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:25.942723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc9e50-d7f0-483e-bc7e-829e5a1b6d7b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:25.965254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:25.992340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:25.993014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:25.993299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:25.993575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:25.994568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:25.994905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:25.996554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b9e31-238d-4ef6-93f7-f6c8fc78e937 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:26.006858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 433/869] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:26.022077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1f56-4d23-43d0-b45e-055356776b44 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:26.061258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:26.062258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:26.062734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:26.063263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:26.064401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=eae0222c39364fa69be2a3cc938516bd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:26.064957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:26.066639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276facb-6d65-455e-a050-a76c1228eb23 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:26.074162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 437/870] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:25 2022] GET /identity/v3/role_assignments?user.id=eae0222c39364fa69be2a3cc938516bd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:27.066564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815f44-06a7-4379-a732-052df090bf24 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.067147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815f44-06a7-4379-a732-052df090bf24 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.067147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815f44-06a7-4379-a732-052df090bf24 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.068735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 434/871] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:27.074052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.074437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.074641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.092564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:27.093748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:27.107955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:27.166357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 438/872] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:27.189893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815f44-06a7-4379-a732-052df090bf24 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.315040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815f44-06a7-4379-a732-052df090bf24 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:27.350520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:27.351280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.351563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.351836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.362868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:27.370904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:27.371832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:27.373591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 435/873] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:27.380509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.490573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24f329-77ef-4124-9bf9-1b72483e1a70 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:27.525949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:27.526626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.526945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.527225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.527980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:27.528289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:27.529697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3016-e08e-4eae-9115-9758e322aee0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:27.538372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 439/874] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:27.543918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.589919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:27.652548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96a813-2763-46f8-9698-c944f723ea4f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:27.672007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647605-0c49-4026-be4a-f87ac93371e1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.672316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647605-0c49-4026-be4a-f87ac93371e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.672581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d647605-0c49-4026-be4a-f87ac93371e1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.674292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 440/875] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:27.681473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.681763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.682037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.692419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:27.693518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.694023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.694554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.699482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:27.700670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:27.704259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:27.707572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:27.709185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:27.727284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:27.728412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:27.729538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:27.729538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:27.729538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:27.729538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:27.736155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 436/876] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:27.737506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.776403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 441/877] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:27.798869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.842014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:27.853828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaba5-8ea0-48a5-a6cb-da5cfae5ff26 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:27.899082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:27.900057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.900354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.900913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.901655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:27.901968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:27.902509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a68f2f-e34e-4c18-99d5-2a0000286f85 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:27.903428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:27.919014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 437/878] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:27.925047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:27.943657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:27.944624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:27.945250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:27.945729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:27.946762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:27.948371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569eea2-cd86-4049-bd3c-3162f1a077dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:27.957762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:27.969327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 442/879] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:27 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:28.048487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f5b2-1240-4beb-bb81-faa0aa000647 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.091362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0240b1-c0dc-47cf-a04b-26261648c8fd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.091926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0240b1-c0dc-47cf-a04b-26261648c8fd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.091926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0240b1-c0dc-47cf-a04b-26261648c8fd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.093350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 443/880] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:28.095187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.096631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.097114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.097657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.099155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.099450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.099789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.106958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.108970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.120149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:28.121760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:28.125503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:28.126444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:28.131278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 438/881] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:28.133631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.145304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:28.238813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 444/882] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:28.255178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883bae-4a32-476c-b00b-9a3c976dad11 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.259920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.296066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.296751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.297038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.297320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.298160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:28.298476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.299935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fceb0-8f17-47ae-b278-524d1e03ba19 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.323866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 439/883] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:28.325048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87183-887b-4af7-960c-9530b93c51b9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.325339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87183-887b-4af7-960c-9530b93c51b9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.325618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa87183-887b-4af7-960c-9530b93c51b9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.330838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 440/884] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:28.331843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa87183-887b-4af7-960c-9530b93c51b9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.376411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973af26-3a7f-4d16-b873-5cede61aee21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.405689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:28.408042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:28.412144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.412860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.413313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.413745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.414558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.415976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e4ef0-c6bc-4877-9187-b8f276af96eb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.432498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 445/885] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:28 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:28.434193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.434611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.435049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.452519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:28.454054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:28.466711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa87183-887b-4af7-960c-9530b93c51b9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.516642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.517590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.518020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.518474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.554799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 446/886] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:28.575185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.610479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.611956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.688623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:28.703263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0b6c9-e63b-4143-940c-04d8244dffb2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.725067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:28.742412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.743490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.743919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.744365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.745167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.746678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.748868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 441/887] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:28.750903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:28.759565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:28.760802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:28.761769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:28.764637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 447/888] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:28.770431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.860590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2b412-fa74-4089-8840-ba207da2fd84 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.900462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0f567-d721-4f92-a7f5-07e0a8d158b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:28.905381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.906179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.906461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.906738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.920599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.922063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.937771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794148a8-4279-480d-8589-a974384801e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:28.939051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794148a8-4279-480d-8589-a974384801e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:28.939051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794148a8-4279-480d-8589-a974384801e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:28.939570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794148a8-4279-480d-8589-a974384801e7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:28.940588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794148a8-4279-480d-8589-a974384801e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:28.941075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794148a8-4279-480d-8589-a974384801e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:28.942626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794148a8-4279-480d-8589-a974384801e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:28.946271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:28.947910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:28.948850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:28.948850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:28.948850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:28.948850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:28.955100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 442/890] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:28.955100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.955314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 448/890] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:28.959972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794148a8-4279-480d-8589-a974384801e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:28.973132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:29.016108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:29.070652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794148a8-4279-480d-8589-a974384801e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:29.090404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d99d-16a5-4c20-9f88-5109ec00c1f7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:29.107491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:29.108868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:29.109375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:29.109869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:29.110815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:29.111326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:29.112890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:29.125968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 449/891] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:29.131777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:29.132453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:29.132732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:29.133008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:29.133753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:29.134061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:29.135520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:29.154633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 443/892] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:29.156235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:29.275490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fa1d-bc29-4c60-9698-e92106fb6413 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:29.290826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:29.314431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:29.315258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:29.315538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:29.315828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:29.322542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:29.324006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:29.338953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:29.339875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:29.340937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 450/893] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:29.347936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:29.429983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:29.456994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba87f9-15aa-4963-b2d2-414242e543f2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:29.499207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:29.500324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:29.500815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:29.501295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:29.502296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:29.502856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:29.504509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066ae61-aa09-4e78-8147-74b899e1b079 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:29.522831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 444/894] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:29.524445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:29.633383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3d7b7-c828-4a94-b2ff-68389343059c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:29.670158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:29.671186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:29.671679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:29.672160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:29.673210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] RBAC: Adding query filter params (role.id=bfdadde65b63412ca262dd804683eb22, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:29.673773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:29.675447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7e370-aa19-4c59-8ee8-a3b29f3a33e5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:29.686529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 451/895] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:29 2022] GET /identity/v3/role_assignments?user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=bfdadde65b63412ca262dd804683eb22 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:30.094290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4c3e1-8ad7-4bdc-a311-b95c675c09e2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.094961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4c3e1-8ad7-4bdc-a311-b95c675c09e2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.095537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4c3e1-8ad7-4bdc-a311-b95c675c09e2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.097353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 445/896] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:30.102793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.103414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.103898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.121694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:30.124262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:30.155882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:30.197087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 452/897] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:30.220141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4c3e1-8ad7-4bdc-a311-b95c675c09e2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.331556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4c3e1-8ad7-4bdc-a311-b95c675c09e2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:30.368128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:30.368940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.369224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.369502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.370161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:30.371643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:30.375074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:30.390587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:30.391678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:30.392499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 446/898] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:30.403953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.508214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146a122-a752-4cc8-ac24-2982308223b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:30.547176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:30.547838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.548122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.548399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.549105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:30.549414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:30.550791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee39ed7-904e-4b28-9b47-3cc27452024f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:30.555868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 453/899] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:30.561664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.580691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842903c-c59a-49d9-9934-1a4028889c0e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.580999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842903c-c59a-49d9-9934-1a4028889c0e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.581506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842903c-c59a-49d9-9934-1a4028889c0e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.583765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 454/900] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:30.590564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.591098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.592276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.614316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:30.616975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:30.655278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:30.669868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b667f-c889-418e-8b20-64c3765d667f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:30.706801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 455/901] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:30.718406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:30.719302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.719733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.720187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.721044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:30.721497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:30.723186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:30.729068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 447/902] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:30.734130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.735234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.805538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:30.852748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db30b1-fe63-4dcc-9609-59e6b3cf360d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:30.864080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693ffc3-41fc-4c03-88d3-0c503d855327 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:30.890834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:30.891803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.892273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.892762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.893942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] RBAC: Adding query filter params (interface=public, service_id=3e8ffb06b53149dda2d1a0d15e419b05, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:30.894467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:30.896175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:30.905689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 448/903] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/endpoints?service_id=3e8ffb06b53149dda2d1a0d15e419b05&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:30.907620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:30.908667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:30.909154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:30.909643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:30.911249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.913992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:30.915690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:30.929203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:30.930171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:30.931063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:30.938069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 456/904] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:30.940848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:30.975241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:31.034959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867899f8-bcee-460b-b701-b28615dedeec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.057170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6200-2c13-4673-86df-93e43aaf9906 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.063224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:31.074741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.075705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.076724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.077223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.078226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.079823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.086747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 449/905] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 06:56:31.100666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.101786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.102302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.102817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.103775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:31.104299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.106099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a274-8ddd-475d-aa00-f0ec8d1075ea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.112505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 457/906] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:31.118616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:31.242810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a38fc9-f393-4121-b2fb-f7dca84d50fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.279328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.280000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.280287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.280566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.281278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:31.281588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.282979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.286480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa94-b82f-4e55-9ce1-419fbc2153cc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.287398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa94-b82f-4e55-9ce1-419fbc2153cc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.287893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2fa94-b82f-4e55-9ce1-419fbc2153cc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.288808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 450/907] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:31.290996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 458/908] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:31.294407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:31.295760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.296257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.296737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.314196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:31.315784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:31.365803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:31.398897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 459/909] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:31.417309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:31.419470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff95ec-0876-4600-b21b-3b3c09105b4f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.450966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:31.470011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.471025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.471514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.472034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.473093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] RBAC: Adding query filter params (interface=public, service_id=b055da914aa14ce29201ceb426c0baf3, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:31.473653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.476219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.483988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 451/910] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:31 2022] GET /identity/v3/endpoints?service_id=b055da914aa14ce29201ceb426c0baf3&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:31.489231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:31.561743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6cc0b-2920-437a-88f2-4f8fdbe998d5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.617079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.618223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.618713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.619374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.620315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.622099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f3fe4-ec9e-497d-ad32-d6ad14ec4a8c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.625555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef4f4b-e2e7-4bb9-9f52-6ca5d5286971 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:31.639398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 460/911] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:31 2022] POST /identity/v3/services => generated 241 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:31.654544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:31.683430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:31.684628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.685120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.686281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.687270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:31.689026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bcac2-18ae-492b-b3cd-11d6dfd00aab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:31.696897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 452/912] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:31 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 06:56:31.760624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48162c-c3a8-46ab-8caf-603ac0e757c8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.760929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48162c-c3a8-46ab-8caf-603ac0e757c8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.761099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48162c-c3a8-46ab-8caf-603ac0e757c8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.762876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 461/913] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:31.768868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:31.769506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:31.770006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:31.789424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:31.791187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:31.869703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 453/914] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:31.892600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48162c-c3a8-46ab-8caf-603ac0e757c8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:31.979145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:32.003959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48162c-c3a8-46ab-8caf-603ac0e757c8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.039940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.041114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.041631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.042111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.053466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:32.062199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:32.063165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:32.064002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:32.066685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 462/915] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:32.079031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.182655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d7ee-454e-497b-9543-a927e95a29ae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.193876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:32.221054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.221737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.222024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.222305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.223094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:32.223407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:32.224785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:32.234365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 454/916] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:32.240457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.348064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6be6-27e4-48cf-b814-08e45acc0da2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.381841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.382636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.382959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.383242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.391027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:32.395468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:32.396960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:32.415717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:32.416670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:32.418018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:32.418018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:32.418018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:32.418018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:32.418018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 463/917] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:32.423078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.526266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e3e4-4b09-4005-bd1d-994f2347045b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.566806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.566806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.566806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.566806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.567227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:32.567227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:32.567227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:32.581021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 455/918] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:32.586649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.711822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd926869-bc21-43e3-be74-b7316560c853 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.745790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.746637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.746965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.747285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.755610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:32.757056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:32.771950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:32.772934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:32.774299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 464/919] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:32.779674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.835150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:32.901719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c130d-aecc-4af6-a490-a3ea9005b9f7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:32.948622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:32.949307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:32.949596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:32.949881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:32.950794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:32.951560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:32.953017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:32.977237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:32.987345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 456/920] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:33.072186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.102102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede2ec2-ef2f-4f76-8501-fb3233d8249d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.149277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.150070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.150358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.150641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.166630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.168116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:33.189117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:33.190095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:33.191100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:33.191100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:33.191100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:33.191100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:33.191100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 465/921] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:33.196810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.315937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.329349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1f5bb-ad35-4798-ad16-a51d5f3a0d0a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.364250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.364921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.365210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.365490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.366217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:33.366527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.367924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc02fe-85d7-4405-8b6a-5aaadbba856f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.380461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 457/922] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:33.385980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.419259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600289a-ffd2-4b9f-8c9a-1e0f69840d53 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.419609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600289a-ffd2-4b9f-8c9a-1e0f69840d53 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.419923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600289a-ffd2-4b9f-8c9a-1e0f69840d53 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.422536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 458/923] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:33.429863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.430444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.430637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.453357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:33.455328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:33.502849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71c511-e457-4c66-9f71-0855fc4e4faf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.504420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.535937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 459/924] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:33.548538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.549335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.549621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.549899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.556933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.559086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.559737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:33.574377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:33.575341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:33.576825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:33.580872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 466/925] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:33.582082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.654592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.679156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4327c-966f-4627-86af-f48dbd56e2f7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.682704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82e3d0-0758-49ab-9b74-c01d5be2049a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.719318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.720121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.720405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.720678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.721334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.722700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.724247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.724898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.725179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.725450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.726228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:33.726536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.727977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:33.739884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:33.740996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 01 06:56:33.741863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:33.749949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 467/927] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:33.749949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.750314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 460/927] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:33.751348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.775471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.855973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dbdab-2b96-4124-9cf3-dc5a565799f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.856764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf6f14-43e1-43c4-961b-644fcaeddbc1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:33.892451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.892799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:33.893345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.893460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.893621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.893722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.893888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/b73d15dbbe6d4bfc99ea858b7e12c3f8/roles/bfdadde65b63412ca262dd804683eb22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.893998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.894691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:33.895031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.896125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=b73d15dbbe6d4bfc99ea858b7e12c3f8, role_id=bfdadde65b63412ca262dd804683eb22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:33.896390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b254b0-104f-4c87-8abb-90ef8e315954 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.897532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6167153-384b-4872-92cc-37c7253f13a3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:33.901472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 461/928] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:33.903231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb167d-e346-4642-b50c-b713a4a14efa None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.903231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb167d-e346-4642-b50c-b713a4a14efa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.903508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb167d-e346-4642-b50c-b713a4a14efa None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.905236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 462/929] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:33.906748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121b9-3f77-4e37-a6ce-4af549fb91de None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.907059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121b9-3f77-4e37-a6ce-4af549fb91de None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.907336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7121b9-3f77-4e37-a6ce-4af549fb91de None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.908609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 463/930] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:33.909492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7121b9-3f77-4e37-a6ce-4af549fb91de None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:33.916032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 468/931] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:33 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/b73d15dbbe6d4bfc99ea858b7e12c3f8/roles/bfdadde65b63412ca262dd804683eb22 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:56:33.917276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863731f-4c08-4020-aafc-d91114f49b60 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:33.917568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863731f-4c08-4020-aafc-d91114f49b60 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:33.917844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863731f-4c08-4020-aafc-d91114f49b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:33.919095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:33.937779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8863731f-4c08-4020-aafc-d91114f49b60 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:33.939285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8863731f-4c08-4020-aafc-d91114f49b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:34.026795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 469/932] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:34.026795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.026795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.026795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.035789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7121b9-3f77-4e37-a6ce-4af549fb91de None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.049563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:34.050859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:34.060021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.079528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.080189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.080470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.081060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.081451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:34.081754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.083146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.088400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 464/933] 10.222.0.48 () {56 vars in 1114 bytes} [Fri Jul 1 06:56:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:34.089243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.133056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 470/934] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:34.133602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.167433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.205752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c418144-5858-4ce1-8c56-e5946335666d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.245180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.246010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.246290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.246564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.247259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.247713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52bbca-ec70-4376-9ae8-cc85f624cbfe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.248642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:34.261689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:34.262666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.263620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 465/935] 10.222.0.48 () {56 vars in 1132 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:34.265208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.288684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.289350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.289630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.289906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.290654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.292025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.307134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.327152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 471/936] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:34.328042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.399253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da80a97-4394-4df7-8c00-580293f5e373 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.443141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.445713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.446683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.447016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.447390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.448527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.450515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:34.468185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:34.469273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:34.470122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 466/937] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:34.475270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.479094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57534038-b09d-4f1b-9eae-697b255de172 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.526101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.526936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.527224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.527502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.529002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:34.529002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.531224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.536526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 472/938] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:34.541622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.598083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302fb96-957e-4400-a8e7-e6c128c56796 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.623268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.632833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.633492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.633781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.634059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.634795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:34.635102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.636465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.642388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 467/939] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:34.647887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.679456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4967df-eaf9-4af7-83a4-3ebbceb613b9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.708958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.726751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.727966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.728250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.728524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.729232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:34.729538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.731832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.739126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 473/940] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:34.749329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.759195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8fd27-c37b-4f13-9e67-aa36daeea636 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.795462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.796435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.796959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.797472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.798354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:34.798873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.800406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.806034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 468/941] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:34.814541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:34.868123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dd0ff-3753-493a-b363-29e9fa458215 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:34.906306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:34.911053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:34.911778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:34.912072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:34.912357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:34.913375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:34.915222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:34.946089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 474/942] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:34.974178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70d34e-bafc-497f-8dce-7b02121fd695 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:35.011145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:35.027196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:35.027887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:35.028184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:35.028467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:35.029312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] RBAC: Adding query filter params (interface=public, service_id=f64c544192ac4a029ab328c76427e685, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:35.029636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:35.031152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6662ca-c46a-4929-a68f-0d3c88534326 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:35.037727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 469/943] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:34 2022] GET /identity/v3/endpoints?service_id=f64c544192ac4a029ab328c76427e685&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:35.043902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:35.159351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da196f-aea1-48c6-8fa0-881290f38b74 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:35.195777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:35.196515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:35.196746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:35.197029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:35.197715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:35.199112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5005d-acd3-4b4c-a6b9-42ab57f1a6cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:35.206521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 475/944] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:35 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 06:56:36.050840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341cf04-b788-4a45-b5ed-6d060fe3a35d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.051371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341cf04-b788-4a45-b5ed-6d060fe3a35d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.051371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341cf04-b788-4a45-b5ed-6d060fe3a35d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.052968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 470/945] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:36.058518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.058960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.059157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.075823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:36.077530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:36.121256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:36.152139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 476/946] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:36.175537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341cf04-b788-4a45-b5ed-6d060fe3a35d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.283259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341cf04-b788-4a45-b5ed-6d060fe3a35d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:36.318039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:36.318793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.319087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.319361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.331347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:36.339273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:36.340426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:36.341330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 471/947] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:36.347453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.449188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e8c18-7c57-4031-b122-7bc6a722acce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:36.484319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:36.484987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.485270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.485545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.486293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:36.486743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:36.488124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1640856-6dec-413b-a42f-30e092a91b11 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:36.496554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 477/948] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:36.502386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.513046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9d2c-d6ed-434b-a632-40d80c35514d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.513335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9d2c-d6ed-434b-a632-40d80c35514d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.514177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9d2c-d6ed-434b-a632-40d80c35514d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.519197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 478/949] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:36.520753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.521043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.521330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.537676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:36.539171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:36.555865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:36.573446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:36.626041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81367850-70e3-425f-9e45-a627c0347a88 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:36.626938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 479/950] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:36.652576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.666701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:36.667528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.667807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.668087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.678596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:36.680078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:36.698418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:36.699379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:36.700275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:36.700275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:36.700275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:36.700275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:36.700275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 472/951] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:36.705896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.771766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55188eb3-68fe-4c77-a7e9-faca46fc59a6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:36.807487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:36.808808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:36.809727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.810185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.821193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84645f87-70c6-4a95-a357-b2e0fa35b7fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:36.821770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.822801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:36.824551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:36.826135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING keystone.server.flask.common [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 01 06:56:36.849344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 06:56:36.849344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 06:56:36.849344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [parameters: {'id': 'e75c980ccf15422b9f915d28e4e41b08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 06:56:36.849344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 06:56:36.853438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:36.854154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:36.854439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:36.854708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:36.855467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:36.855754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:36.857165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e75c980ccf15422b9f915d28e4e41b08.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e75c980ccf15422b9f915d28e4e41b08. Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:56:36.874845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 01 06:56:36.875758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 01 06:56:36.876791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 06:56:36.877695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 06:56:36.879078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:56:36.879872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': 'e75c980ccf15422b9f915d28e4e41b08', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.880869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:36.881760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e75c980ccf15422b9f915d28e4e41b08. Jul 01 06:56:36.882789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:36.883513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 473/952] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:36.883513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.885623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 480/953] 10.222.0.48 () {60 vars in 1140 bytes} [Fri Jul 1 06:56:36 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:36.898813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:36.931541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.006557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c39cb8-24df-4c71-852a-c2ee06f055c8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.032889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc10acd-2e02-4687-850c-1b5ee11f8197 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.048921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.048921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.048921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.048921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.057410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.059293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:37.077415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:37.078658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:37.079853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 474/954] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:37.084031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.089521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.090928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.091433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.091930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.103578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.106248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.111916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:37.144351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:37.146054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:37.148538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 481/955] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:37.153116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.247421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7899d-243a-49e8-9cd9-777ca18a39d5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.280071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.291118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.291964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.292586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.292873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.293874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:37.294198 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.296213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4c5c-33ee-4893-958a-12d85e80dabe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.298892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6493-f359-491f-8156-0c8f9a3a8104 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.317853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 475/956] 10.222.0.48 () {56 vars in 1149 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:37.319461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ce8eb-e4b6-4f2f-abc1-454bd1106227 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.319889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ce8eb-e4b6-4f2f-abc1-454bd1106227 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.320333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ce8eb-e4b6-4f2f-abc1-454bd1106227 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.321762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 476/957] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:37.323216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ce8eb-e4b6-4f2f-abc1-454bd1106227 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.340556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.341263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.341543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.342462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.343245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:37.343553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.345319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11994918-9d4f-4c35-aa0b-ee7ba1402290 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.367359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 482/958] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:37.368718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae54c9-1f71-4d4f-ade1-3d39430fc6cf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.369099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae54c9-1f71-4d4f-ade1-3d39430fc6cf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.369257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae54c9-1f71-4d4f-ade1-3d39430fc6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.398381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-26ae54c9-1f71-4d4f-ade1-3d39430fc6cf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:37.400029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26ae54c9-1f71-4d4f-ade1-3d39430fc6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:37.405860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.446166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ce8eb-e4b6-4f2f-abc1-454bd1106227 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 483/959] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228cdb-b5cb-4c9c-9d50-208ed4a163e6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228cdb-b5cb-4c9c-9d50-208ed4a163e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228cdb-b5cb-4c9c-9d50-208ed4a163e6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 484/960] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.486794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.487439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.498758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.500336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.500732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.501105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.505935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:37.507180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:37.578473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.579953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.586331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 485/961] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:37.587469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.591647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.656729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.701603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 477/962] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:37.701603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.706802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff676ff-f22c-4518-a9f4-13cacf929692 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.756210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.757004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.757279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.757612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.758186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.759596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:37.771389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:37.772446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:37.773298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:37.777498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 486/963] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:37.778359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.803119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44d76-3f34-41b0-9280-380186cd7958 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.838371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.839208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.839490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.839767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.840416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.841863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.851703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:37.853473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:37.854791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:37.855851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:37.860339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 478/964] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:37.861051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:37.894840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:37.895438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a819b0-6292-4c69-a99d-d5b8e99e67f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.931253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:37.932281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:37.932716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:37.933397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:37.946018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:37.947783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:37.962489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43fce5-0dca-419a-8c3d-5dc948d8a534 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:37.965774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:37.967398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:37.968540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:37.968540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:37.968540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:37.968540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:37.970457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 487/965] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:37.971477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.001926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.002608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.002930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.003205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.003911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.004214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.005570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.010278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 479/966] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:38.011060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.086576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f47cecc-b520-415c-9c56-7799d93fabe9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.108860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.120250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.124050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e261c-a209-4b66-a479-1332a274dc9b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.126494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.127194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.127475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.127752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.128455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.128762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.130132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.135455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 488/967] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:38.136083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.166335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.167029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.167312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.167807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.168299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.168658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.170560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.183744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 480/968] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:38.184405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.256020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d6ee3-24e3-442f-b68f-01d2e9021cb8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.291094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.291752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.292037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.292315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.292547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b73e5-01e7-4235-9d16-90126bad73b8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.293023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.293331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.294799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.299421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 489/969] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:38.300025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.351050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.352428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.353485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.354142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.355253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.356233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.356492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.358934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.366046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 481/970] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:38.371666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.403276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.438562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0852749-7426-45e1-964e-b54637fd8172 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.487079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.487434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.487757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.487950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.495999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.498741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.504574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a43996-4427-49f9-8e0a-b5d5e40d4d79 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:38.518263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:38.519415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:38.520375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 490/971] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:38.525635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.550741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.552290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.552856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.553327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.554956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.557125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.590973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 482/972] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:38.643977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6fc3-690d-4ae4-a31b-5b660764bbfc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.655114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.693420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.694197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.694485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.694759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.695619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.695932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.697395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205a2c6-8bda-4b30-80a2-d77bd43d2f88 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.711689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 491/973] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:38.718268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:38.808362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:38.833990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d641-0098-449b-8cf8-b99f576ca765 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:38.871169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:38.871822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:38.872104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:38.872377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:38.873791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] RBAC: Adding query filter params (role.id=bfdadde65b63412ca262dd804683eb22, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:38.874102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:38.875602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2328617-7aad-443e-a8df-563b3d6d19ae None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:38.881999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 483/974] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:38 2022] GET /identity/v3/role_assignments?user.id=b73d15dbbe6d4bfc99ea858b7e12c3f8&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=bfdadde65b63412ca262dd804683eb22 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:39.508013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff461-02b8-47df-b8ee-c604667d0f59 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:39.508524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff461-02b8-47df-b8ee-c604667d0f59 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:39.508524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ff461-02b8-47df-b8ee-c604667d0f59 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:39.509999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 492/975] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:39.515447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:39.516336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:39.516978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:39.538302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:39.540246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:39.614869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 484/976] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:39.634432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff461-02b8-47df-b8ee-c604667d0f59 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:39.739655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ff461-02b8-47df-b8ee-c604667d0f59 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:39.776838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:39.777799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:39.778295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:39.778811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:39.780378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:39.782174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:39.801692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 06:56:39.801692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 06:56:39.801692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [parameters: {'user_id': 'a2ada42cac8b45e18fbcd7da87e42290', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 06:56:39.801692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 01 06:56:39.809613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:56:39.811674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 01 06:56:39.812937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 01 06:56:39.813869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 01 06:56:39.815571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 01 06:56:39.819886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a2ada42cac8b45e18fbcd7da87e42290', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:39.820778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 01 06:56:39.821600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:39.822511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 493/977] 10.222.0.48 () {60 vars in 1131 bytes} [Fri Jul 1 06:56:39 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:39.824227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:39.911477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:39.936743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb7ca2-0e0f-4a72-972d-c71de7a6e5fe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:39.973403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:39.974437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:39.974924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:39.975365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:39.983232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:39.984843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1b386fd4-00a6-4509-a97a-a777cbaed674 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:40.000143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:40.001326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 01 06:56:40.002536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:40.004802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 485/978] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:40.011063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:40.045638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:40.123642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88e96d-9fd2-4eb1-937f-4549dabb08dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:40.159177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:40.160101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.160534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.160958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.161906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:40.162363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:40.164187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a640-6fd8-49cf-8cf6-5502c9a6a5d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:40.203982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 494/979] 10.222.0.48 () {56 vars in 1101 bytes} [Fri Jul 1 06:56:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:40.448761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0c1a9-41c7-4006-b6b2-a6a414c68bf7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.449254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0c1a9-41c7-4006-b6b2-a6a414c68bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.449665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0c1a9-41c7-4006-b6b2-a6a414c68bf7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.451345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 486/980] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:40.456620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b924-7edb-49b5-b256-ae1d46236897 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.456914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b924-7edb-49b5-b256-ae1d46236897 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.457189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b924-7edb-49b5-b256-ae1d46236897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.474481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6f27b924-7edb-49b5-b256-ae1d46236897 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:40.476030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6f27b924-7edb-49b5-b256-ae1d46236897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:40.541859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 495/981] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:40.564219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0c1a9-41c7-4006-b6b2-a6a414c68bf7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:40.665834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0c1a9-41c7-4006-b6b2-a6a414c68bf7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:40.701918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:40.702502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.702807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.703084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.703875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:40.705236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3c919-327b-4544-9de4-9a33eb4b2f79 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:40.707400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:40.730805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 487/982] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:40 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:40.820336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c650044-26b4-4d53-a6bb-09799d73e02d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.820710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c650044-26b4-4d53-a6bb-09799d73e02d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.820904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c650044-26b4-4d53-a6bb-09799d73e02d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.822539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 496/983] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:40.828561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdbab-1fe2-48a1-83f8-fee2b17a8928 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.828874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdbab-1fe2-48a1-83f8-fee2b17a8928 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.829132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fdbab-1fe2-48a1-83f8-fee2b17a8928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.847456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b11fdbab-1fe2-48a1-83f8-fee2b17a8928 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:40.848693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b11fdbab-1fe2-48a1-83f8-fee2b17a8928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:40.928493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 488/984] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:40.952895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c650044-26b4-4d53-a6bb-09799d73e02d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:40.968922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:40.974499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893d6f-7730-463a-b269-72e9f9cce7c1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.974812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893d6f-7730-463a-b269-72e9f9cce7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.975097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29893d6f-7730-463a-b269-72e9f9cce7c1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.976606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 489/985] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:40.981593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:40.981943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:40.982280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:40.999695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:41.000886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:41.059351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c650044-26b4-4d53-a6bb-09799d73e02d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.071457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 490/986] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:41.093838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.095304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.096113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.096395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.096666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.097316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.098689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:41.110802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:41.111800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:41.112697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 497/987] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:41.117734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.149203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:41.204773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:41.213532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3c2ba-84b7-4fdb-bc47-e676bae81bc6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.226997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f1c3-c273-4f08-8e19-0e47cac8c1c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.252882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.253674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.253963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.254429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.254938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.256308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.265165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.265828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.266110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.266386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.267123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:41.267433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:41.267916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:41.268895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:41.269745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:41.270503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.270573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 491/988] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:41.275227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.283303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 498/989] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:41.284365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.384439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37b777-18b6-4c4f-943d-b06aa41adcb0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.387322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf9856-a06e-4cc7-992b-e0ee5ca9e203 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.399349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:41.420584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.421232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.421516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.421794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.422499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] RBAC: Adding query filter params (name=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:41.422842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.424195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.425454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.426119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.426411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.426690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.427429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:41.427739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.429090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.434186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 492/990] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:41.435020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.438809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 499/991] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:41.443966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.453176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:41.547628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc36fc-e668-46cc-b28a-93c64fbd735a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.553263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6288-781e-444a-af6c-2e1e7f17fe36 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.582759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.583435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.583719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.584251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.584839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] RBAC: Adding query filter params (interface=admin, service_id=f64c544192ac4a029ab328c76427e685, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:41.585106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.586455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.588993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.589647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.589930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.590205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.590935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:41.591246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.592989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 500/992] 10.222.0.48 () {56 vars in 1249 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/endpoints?service_id=f64c544192ac4a029ab328c76427e685&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:41.593289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10a28e-8c06-4dbd-8cff-36ec448e4da0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.606936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 493/993] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:41.607226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:41.681553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:41.716919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eaec3-435b-4f68-8ac8-5045735341f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:41.752044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:41.752712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:41.752989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:41.753260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:41.754815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:41.755335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd61df-2a1b-4d5d-a71c-db68172fdbdf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:41.764132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 501/994] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:41 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 06:56:42.771630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352077cc-2a15-48fc-a21b-f3f33c5fc820 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:42.772150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352077cc-2a15-48fc-a21b-f3f33c5fc820 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:42.772260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352077cc-2a15-48fc-a21b-f3f33c5fc820 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:42.773570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 494/995] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:42.778488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:42.778923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:42.779709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:42.795878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:42.797042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:42.866630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 502/996] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:42.889937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352077cc-2a15-48fc-a21b-f3f33c5fc820 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:42.935693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:42.994839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352077cc-2a15-48fc-a21b-f3f33c5fc820 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.027174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.027891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.028169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.028441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.029632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.031011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:43.042319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:43.043340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:43.044211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:43.045546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 495/997] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:43.049732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.110320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:43.148691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9a520-f67d-4d08-adc1-235f945c06b5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.181112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.181767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.182093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.182369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.183104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:43.183680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.185031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.198953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 503/998] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:43.200590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.305824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b097f-5d72-48e4-ab7d-eca5398ccccc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.313676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:43.342188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.342882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.343170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.343449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.344157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:43.344461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.345823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.351001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 496/999] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:43.356627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.457620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310bcc-6739-44e9-ad07-69f8e8f97d22 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.483603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:43.492125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.492782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.493064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.493341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.494129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] RBAC: Adding query filter params (interface=public, service_id=29210672406b4d70be8ecb1306b74b3f, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:43.494444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.495830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1020c55-c8d2-4fa8-a614-7c1ab58c367b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.505974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 504/1000] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:43 2022] GET /identity/v3/endpoints?service_id=29210672406b4d70be8ecb1306b74b3f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:43.507224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.611103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29016d-2242-4ab9-8b55-26e50f3040a4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.644509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.645176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.645462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.645741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.646428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.647821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32317d57-ef9e-4457-a66f-e487503d1271 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.662812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 497/1001] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:43 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 06:56:43.672543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964a0c2-566c-4646-9548-491f23b85407 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.672835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964a0c2-566c-4646-9548-491f23b85407 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.673112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964a0c2-566c-4646-9548-491f23b85407 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.674725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 505/1002] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:43.680290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.680623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.680859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.699401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:43.700616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:43.729247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:43.774714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 498/1003] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:43.794298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13237e-991c-446a-902e-309a4598c3e6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.795419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13237e-991c-446a-902e-309a4598c3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.795517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a13237e-991c-446a-902e-309a4598c3e6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.797213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 506/1004] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:43.798432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.803019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.803315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.803579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.821139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:43.822533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:43.896644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 507/1005] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:43.905963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5812-39d7-4d9c-bb73-a4cf1cb51bbf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:43.920352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:43.943556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:43.944220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:43.944500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:43.944773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:43.945497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:43.946902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:43.962564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 499/1006] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:43 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:44.035236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6db3a8-587d-4de6-9eee-07315d43ab5f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:44.061054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:44.080627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:44.081450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:44.081736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:44.082014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:44.082679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:44.084196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:44.097600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:44.098678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:44.099595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 508/1007] 10.222.0.48 () {56 vars in 1126 bytes} [Fri Jul 1 06:56:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:44.104930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:44.134897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:44.220986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031938ad-69b6-4cbe-85ea-8510ac6bc400 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:44.256017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:44.256683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:44.256968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:44.257246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:44.258020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:44.258330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:44.259789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:44.269658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 500/1008] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:44.271172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:44.372437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68642fe-633c-4315-8da0-ce256f77583c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:44.406913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:44.407574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:44.407856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:44.408134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:44.408842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:44.409151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:44.410512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60913cd2-5c55-440f-a7d2-db1334e16e76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:44.418917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 509/1009] 10.222.0.48 () {56 vars in 1122 bytes} [Fri Jul 1 06:56:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:44.421591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:44.512319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:44.525600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0323df4-8b6c-4422-9e85-b7485b9990d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:44.560122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:44.560807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:44.561090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:44.561366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:44.562097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:44.563484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757414b5-633d-4d4f-a8bc-63b0f4757ebc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:44.578851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 501/1010] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:45.720080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd887-1564-4a2c-af76-7bcaed648e8d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:45.720628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd887-1564-4a2c-af76-7bcaed648e8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:45.720628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580bd887-1564-4a2c-af76-7bcaed648e8d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:45.722040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 510/1011] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:45.727094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:45.727508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:45.727680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:45.744745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:45.745925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:45.818095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 502/1012] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:45 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:45.840739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bd887-1564-4a2c-af76-7bcaed648e8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:45.889978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:45.945925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580bd887-1564-4a2c-af76-7bcaed648e8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:45.978655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:45.979415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:45.979697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:45.979971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:45.980612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:45.981968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ea54adf2-6ac6-4a6a-9909-0482dabb4312 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:45.994122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:45.995357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 01 06:56:45.996201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:46.001243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 511/1013] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:46.002374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.017880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761053e-31ed-4768-b367-f2a27fe03ef4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.018174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761053e-31ed-4768-b367-f2a27fe03ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.018715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761053e-31ed-4768-b367-f2a27fe03ef4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.020424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 512/1014] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:46.025502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.025791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.026067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.047082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:46.048919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:46.078746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.106100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5ce83-346b-42ec-b117-034584993452 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.129630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 513/1015] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:46 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:46.143267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.144166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.144624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.145082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.145956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:46.146448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.148002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.151710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.153407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 503/1016] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:46.160210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.198627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.264311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126e67e-3fa5-4409-8ea7-4980012283ee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.270587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058889d7-e009-4699-9e09-4cdd03527f7f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.300177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.300962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.301244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.301521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.302234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.303562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.305448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.306109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.306388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.306664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.307396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:46.307700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.309308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.314267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 504/1017] 10.222.0.48 () {56 vars in 1112 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:46.315559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:46.316529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:46.322541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 514/1018] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:46.323416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.323499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.325544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.426954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67435ccc-b3cf-44d6-a4a7-dd4325afe9d0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.430415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.442907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61add88-d644-48ec-81e0-ede5bc546f52 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.464761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.465661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.465942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.466215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.467501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:46.467803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.469162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.478565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 515/1019] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:46.479667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.485149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.486213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.486707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.487232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.488191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.489730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a77fe-1104-4339-bbaf-6fa442a96263 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.510359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 505/1020] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:46.597258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8b677-af9a-4f01-af0b-73e974979a35 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.621601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365877b1-e74d-40dd-8dd8-f2259cbb7b1e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.621897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365877b1-e74d-40dd-8dd8-f2259cbb7b1e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.622177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365877b1-e74d-40dd-8dd8-f2259cbb7b1e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.623806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 506/1021] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:46.629237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.629564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.629801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.641354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.642030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.642311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.642584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.643334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:46.643645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.645028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.647883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:46.649284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:46.650476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 516/1022] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:46.657034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.692510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.718699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.735961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 507/1023] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:46.758809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.781198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3b8c-3ee1-4e85-a143-48dfa6b0ea58 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.816638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.817651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.818148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.818609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.819591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] RBAC: Adding query filter params (interface=public, service_id=0d947293cebe4b72ad2a9a1f17edf5d8, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:46.820107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.821648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.827936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 517/1024] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/endpoints?service_id=0d947293cebe4b72ad2a9a1f17edf5d8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:46.832932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.872485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032daa0-15e2-40e6-ab40-6599b9f1e7bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.908673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.909465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.909749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.910024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.911243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.912664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.914486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 508/1025] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:46.920109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:46.930272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.942171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d8f80-9529-435e-9612-7f6a7eaea8de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:46.977201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:46.977815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:46.977952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:46.978148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:46.978426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:46.979147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:46.980512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdfcd2-49ac-4e90-baba-eb4cbc2a8def None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:46.987491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 518/1026] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:56:46 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 06:56:47.038643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529dbbc-1eae-412b-91d4-519b06875e93 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:47.077347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:47.078032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:47.078319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:47.078597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:47.079395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:47.080863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11a5e6-0fab-4c7f-8994-4a53dde80f78 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:47.114800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 509/1027] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:56:46 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:56:48.662930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664edb7c-bb41-43a5-812c-19fc1dd580e9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:48.663474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664edb7c-bb41-43a5-812c-19fc1dd580e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:48.663474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664edb7c-bb41-43a5-812c-19fc1dd580e9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:48.665065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 519/1028] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:48.670046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:48.670423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:48.670604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:48.688022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:48.689194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:48.757326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 510/1029] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:48.780543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664edb7c-bb41-43a5-812c-19fc1dd580e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:48.884581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664edb7c-bb41-43a5-812c-19fc1dd580e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:48.918103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:48.918847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:48.919130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:48.919405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:48.929949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-24e5695f-669b-4845-a23c-55b32a85b8be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:48.938829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:48.939811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:48.940709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:48.942238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 520/1030] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:48.946246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:48.984728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.039246 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8eb92-0c4d-4360-bc52-7ca926277918 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.039590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8eb92-0c4d-4360-bc52-7ca926277918 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.039799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8eb92-0c4d-4360-bc52-7ca926277918 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.041306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 521/1031] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:49.046175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72aa578-0cfa-4c8c-8b3b-0b5a2f164723 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.046372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.046639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.046950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.063940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:49.065106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:49.076132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.084437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.085109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.085391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.085665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.086514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:49.086851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.088213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.097046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 511/1032] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:49.102578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.138579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 522/1033] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:49.161776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.206890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.213395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb5d8-da69-40bd-8d7a-8ef26b872d11 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.247090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.247911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.248194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.248477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.258613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.260068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.262435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3023-80c7-4f70-9344-8c67078459c8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-02c0a6ba-8fe1-4684-808b-487e6099566f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:49.276793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:49.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:49.278728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:49.278728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:49.278728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:49.278728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:49.278728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 512/1034] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:49.279869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80984-c133-4a06-b218-eca06ad2877e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.280157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80984-c133-4a06-b218-eca06ad2877e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.280439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b80984-c133-4a06-b218-eca06ad2877e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.281691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 513/1035] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:49.284214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b80984-c133-4a06-b218-eca06ad2877e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.301268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.302046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.302332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.302610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.303283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.304636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a19ff4f5-fc8a-46f8-a018-1e5ebe018ff6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:49.316025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:49.317065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:49.317947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 523/1036] 10.222.0.48 () {56 vars in 1105 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:49.318749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.318749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.318923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.335008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:49.336182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:49.338858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.387243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b80984-c133-4a06-b218-eca06ad2877e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.402220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 524/1037] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:49.403061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.421374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.422032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.422310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.422584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.423331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:49.423634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.424986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.441025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 514/1038] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:49.441025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.455740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.516828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cee77-a94d-41f1-bf47-26c42c5f5876 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.538154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe5f2-5fb8-4c2c-a877-6db639e0921e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.549722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.550375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.550662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.550963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.551647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:49.551959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.553310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.558110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 525/1039] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:49.559160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.564295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.573533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.574340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.574620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.574921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.576127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.577522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.579158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 515/1040] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:49.579900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.657847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.662125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6426ca-39b4-433b-a76e-de988625f814 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.677081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5b6d0-d77f-4244-8e07-d53d65595613 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.697987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.698787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.699081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.699363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.707219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.708704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.710205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.710891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.711176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.711448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.712147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:49.712452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.713783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.718574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 516/1041] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:49.719285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:49.722227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:49.723252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 526/1042] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:49.724210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.785023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.821291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cdfd0-8916-4d8f-8d5b-37a207437181 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.822982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaaab3-3589-4f8c-9854-150d31705258 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:49.853574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.854228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.854514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.854711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:49.854941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.855381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:49.855520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.855664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:49.855942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:49.856666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:49.856950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.858549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:49.872694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:49.877821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 527/1043] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:56:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:49.878612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:49.883571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 517/1044] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:56:49 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:56:50.005545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe4275-fc23-4e0d-8d88-d0404801cfe1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:50.024814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:50.050213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:50.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:50.051441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:50.051710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:50.052549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:50.052856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:50.054276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:50.072557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 528/1045] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:50.078536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:50.185605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828b568-7527-4ca3-8f44-19e36bd10643 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:50.225173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:50.225824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:50.226108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:50.226386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:50.292348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:50.295434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:50.296893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:50.398755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 518/1046] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 06:56:50.405170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:50.506064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c0fd-3e10-459e-8e40-0ce045b9bba0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:50.539970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:50.540759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:50.541037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:50.541309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:50.551217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:50.552652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:50.569728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:50.570631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:50.571863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:50.571863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:50.571863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:50.571863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:50.572435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 529/1047] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:50.577237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:50.676759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:50.680520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bd4cf-6516-4fd0-8287-6082703a5e88 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:50.716604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:50.717260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:50.717545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:50.717824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:50.718543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:50.718878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:50.720247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:50.735841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 519/1048] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:50.737161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:50.766658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:50.842689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37074c-0975-4519-aba8-7e452a60e271 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:50.880332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:50.881117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:50.881403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:50.881734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:50.888999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:50.890452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:50.903998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:50.904889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:50.905855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 530/1049] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:50.910876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:51.015608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01de1ad-8ed9-42fa-9d26-733b5807578f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:51.051764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:51.052431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:51.052714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:51.052990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:51.053782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:51.054093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:51.055548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bb9fe-d1a8-44e3-a0f2-2fdc0794cf15 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:51.068615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 520/1050] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:51.074844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:51.161873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:51.178457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f361d-5b6a-407b-84c8-a06fbf81c6b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:51.219066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:51.219725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:51.220010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:51.220288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:51.221112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] RBAC: Adding query filter params (role.id=bfdadde65b63412ca262dd804683eb22, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:51.221428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:51.222868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d241bb-2546-42f0-85e4-efc1f027fa4d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:51.238842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 531/1051] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:51 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=bfdadde65b63412ca262dd804683eb22 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:51.968463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d25765-632a-4c11-8a3b-5dc446af86b7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:51.968988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d25765-632a-4c11-8a3b-5dc446af86b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:51.968988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d25765-632a-4c11-8a3b-5dc446af86b7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:51.970406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 521/1052] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:51.975437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:51.975845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:51.976014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:51.992287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:51.993468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:52.014995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c09d-5ab6-40d7-9e9a-ac1b7edc83ca None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.015281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c09d-5ab6-40d7-9e9a-ac1b7edc83ca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.015555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c09d-5ab6-40d7-9e9a-ac1b7edc83ca None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.017030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 522/1053] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:52.021727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.022018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.022300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.039061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:52.040258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:52.054146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.064523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 532/1054] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:52.086763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.111688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 523/1055] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:52.135578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.170079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.194112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5c3f2-f418-4e36-a046-7d5fffc2a1af None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.229611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.230434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.231123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.231408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.245402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5c1bd-1d02-4b93-b624-fb0fc59cb678 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.283396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.284188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.284468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.284749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.296189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:52.304116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:52.305219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:52.306070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 524/1056] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:52.312238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.314486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.315997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.325411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.402596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.416787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be1c9-0a26-4de6-97df-7ebd73e3b848 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.427139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 533/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 06:56:52.429903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.453726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.454406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.454690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.454991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.455734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:52.456041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.457412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.470870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 525/1058] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:52.472824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.540286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cdc1e-49d3-4a15-b0c8-d20e7e98605e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.543563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.575494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba874a-4e79-4263-bf2b-ff426d6359ab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.576501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.577400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.577788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.578068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.589074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.591339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.609881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.610741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.611047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.611321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] PATH_INFO: `/v3/users/126ceafcef354edc8d835246b37ecb8b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:52.613329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:52.614188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:52.615149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:52.615149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:52.615149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:52.615149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:52.615149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 534/1059] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:52.621806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.629208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] RBAC: Authorizing `identity:get_user(user_id=126ceafcef354edc8d835246b37ecb8b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.631078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.637908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 526/1060] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/users/126ceafcef354edc8d835246b37ecb8b => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:52.638972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:52.652825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.727690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1d7e-0dcd-420c-95a8-02ba0a8d64f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.740639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aff322-5928-4efe-a124-e1b40edb402f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:52.764038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.764697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.764987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.765268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.766106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:52.766417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.767949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.768919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:52.775565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:52.776211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:52.776494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:52.776773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:52.781344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 535/1061] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:52.862786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:52.864293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:52.973072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 527/1062] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:52.979382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.083953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:53.093607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a18c1c-e5b2-4a64-bbfa-4f58212cd08e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.132234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.133016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.133291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.133565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.134723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:53.136170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58e9e3-935b-406e-9b57-5e2d01cb5bac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:53.137855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 536/1063] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:56:52 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:53.143875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.173600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:53.242889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5279-5307-432d-96c7-3f17bfef41bb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.255151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e890-6dc5-4a5d-a1e1-d2b403913079 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.255435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e890-6dc5-4a5d-a1e1-d2b403913079 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.255709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896e890-6dc5-4a5d-a1e1-d2b403913079 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.257081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 537/1064] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:53.261860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.262147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.262453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.276765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.277418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.277703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.277984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.278871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=cdc0d36b83844a979be5e2535b68274c, user.id=126ceafcef354edc8d835246b37ecb8b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:53.279692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:53.279815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:53.280991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:53.281093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:53.286574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 528/1065] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:53 2022] GET /identity/v3/role_assignments?user.id=126ceafcef354edc8d835246b37ecb8b&scope.project.id=cdc0d36b83844a979be5e2535b68274c&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:53.358193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 538/1066] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:53.381609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.478520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:53.487650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505dc86-aec6-4a07-bca9-5be0c42212a3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.522525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.523356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.523640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.523916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.534370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:53.542416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:53.543547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:53.544420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 529/1067] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:53.549812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.637725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:53.648406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e90f0f-2127-4b26-ae73-d76f04eec9d4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.680837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.681500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.681785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.682060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.682824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:53.683136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:53.684758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:53.693036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 539/1068] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:53.698720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.797536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2128732-3509-406d-aced-ade43aec03fe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.830755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.831556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.831838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.832112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.842237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:53.843695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:53.860021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:53.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:53.861984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:53.861984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:53.861984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:53.861984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:53.861984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 530/1069] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:53.866937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:53.964068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdaed4-b170-4a71-a273-6575f5f38a32 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:53.995954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:53.996762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:53.997048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:53.997324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:53.998658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:53.998993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.000345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.011643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 540/1070] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:54.017173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.026290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.114402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6464-5f80-4edf-aa44-e66d708fc530 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.146287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.147084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.147369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.147643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.154816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.156241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:54.170283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:54.171458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:54.172828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 531/1071] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:54.177064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.192686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.274733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c22a2-0597-4e01-842f-49a422cc4006 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.307897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.308813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.309095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.309373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.310749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:54.311100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.312505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.328956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 541/1072] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:54.334552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.399680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.434181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584bf7e2-0da2-4452-9ede-e7b05e8df184 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.465847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.467145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.467424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.467690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.477753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.479216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:54.496261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:54.497351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:54.498548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:54.498548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:54.498548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:54.498548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:54.502031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 532/1073] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:54.503134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.570248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.600280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6caa97-267f-4ce3-9bd5-86f0c175f9ce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.632752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.633401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.633688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.633964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.634679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:54.635027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.636389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.647252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 542/1074] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:54.652762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.750629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb0087-5b20-48f3-9c1d-b4898e08c759 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.767327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.784254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.785043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.785327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.785604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.792192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.793619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:54.807469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:54.808801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:54.809728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 533/1075] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:54.814687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:54.938340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e021eb-b595-4676-973b-7af8d357d045 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:54.967200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:54.973435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:54.974212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:54.974402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:54.974677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:54.975503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:54.975812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:54.977271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae921206-6daa-4b24-a888-c3686d6df930 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:54.989195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 543/1076] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:54.995101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:55.091217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcf850-b07a-48e7-a324-57c4670a2047 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:55.124341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:55.125215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.125495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.125769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/bfdadde65b63412ca262dd804683eb22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.127968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=458dc4b52e3d4a13a8a4f801ddf2e6c4, role_id=bfdadde65b63412ca262dd804683eb22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:55.129394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65cc8c2-9429-4340-8180-c5637747e2ef None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:55.142696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 534/1077] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:54 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/458dc4b52e3d4a13a8a4f801ddf2e6c4/roles/bfdadde65b63412ca262dd804683eb22 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 06:56:55.290927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665204-4654-476e-9ebe-ac3d0bbf1caa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.291483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665204-4654-476e-9ebe-ac3d0bbf1caa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.291483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70665204-4654-476e-9ebe-ac3d0bbf1caa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.292899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 544/1078] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:55.298374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.298718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.300967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.318063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:55.319285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:55.349043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:55.387136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 535/1079] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:55.409658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665204-4654-476e-9ebe-ac3d0bbf1caa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:55.510356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70665204-4654-476e-9ebe-ac3d0bbf1caa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:55.544561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:55.545334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.545613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.545882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.555791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:55.563545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:55.564503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:55.565538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 545/1080] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:55.570847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:55.667606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109695-7b83-4eda-99e3-57660c7e3651 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:55.699820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:55.700477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.701291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.701567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.702305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:55.702604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:55.703989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:55.711843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 536/1081] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:55.718294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:55.766385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:55.817475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9043-f175-441b-9a4d-49022959b448 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:55.851029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:55.851890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.852167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.852438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] PATH_INFO: `/v3/users/126ceafcef354edc8d835246b37ecb8b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.853612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] RBAC: Authorizing `identity:get_user(user_id=126ceafcef354edc8d835246b37ecb8b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:55.855060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:55.856825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 546/1082] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:56:55 2022] GET /identity/v3/users/126ceafcef354edc8d835246b37ecb8b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:55.862323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:55.892900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:55.961576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d134e7-7612-44e3-a9eb-ed8ee2730c08 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:55.992980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:55.993848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:55.994127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:55.994398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:55.995668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:55.997225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3736127-43c9-42f4-abff-ea5affa2d621 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:55.999664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 537/1083] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:56:55 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:56.005224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:56.101762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc55ad-5429-4bd9-bdc8-e7dd8d7ab24a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:56.119075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:56.136031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:56.136924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:56.137189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:56.137462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c/users/126ceafcef354edc8d835246b37ecb8b/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:56.139992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc0d36b83844a979be5e2535b68274c, user_id=126ceafcef354edc8d835246b37ecb8b, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:56.141401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d07aee-e137-45f3-a012-4683f161532c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:56.153274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 547/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:56:56 2022] PUT /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c/users/126ceafcef354edc8d835246b37ecb8b/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 06:56:57.195982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698544-2b68-4751-92b9-302f1d98b81b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.196581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698544-2b68-4751-92b9-302f1d98b81b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.196581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698544-2b68-4751-92b9-302f1d98b81b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.197877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 538/1085] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:56:57.203537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.203940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.204094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.221229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:57.222428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:57.288318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 548/1086] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:56:57.311989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e698544-2b68-4751-92b9-302f1d98b81b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:57.411802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:57.418019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e698544-2b68-4751-92b9-302f1d98b81b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:57.452079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:57.452814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.453098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.453376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.463583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:57.471663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:57.472768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:57.473633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 539/1087] 10.222.0.48 () {56 vars in 1120 bytes} [Fri Jul 1 06:56:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:57.478838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:57.573152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d09d-75d3-4052-8e7e-2e4f7c25b2ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:57.582643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:57.607913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:57.608576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.608879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.609133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.609873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:57.610179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:57.611572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:57.619340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 549/1088] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:56:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:57.626367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:57.722819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae49ffa-84bd-46de-b468-888e93df83eb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:57.755991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:57.756767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.757049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.757331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.767105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:57.768548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:57.785430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:57.786353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:57.787388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:57.787388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:57.787388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:57.787388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:57.787388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 540/1089] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:57.792416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:57.890018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffc71f-0939-4c8a-ad95-a494b075c9b8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:57.922516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:57.923208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:57.923492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:57.923769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:57.924486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:57.924797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:57.926153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:57.938589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 550/1090] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:57.944152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:57.956181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.041232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fbab7-a560-4f17-9b97-3886ea98df99 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.075279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.076086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.076371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.076646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.083607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.085046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-accc7e0b-8765-4c9a-a038-662234f5eabd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:56:58.099146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:58.100057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:58.100990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 541/1091] 10.222.0.48 () {56 vars in 1093 bytes} [Fri Jul 1 06:56:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:56:58.106168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.136401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.158819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7f067-46fc-4199-9113-0ef95b7cde86 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.159114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7f067-46fc-4199-9113-0ef95b7cde86 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.159394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7f067-46fc-4199-9113-0ef95b7cde86 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.160754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 542/1092] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 06:56:58.165688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.165972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.166251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.183133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:56:58.184597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:56:58.207049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb138ec1-10ce-481a-872f-270f642f56cf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.240293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.240952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.241230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.241503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.242323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:58.242628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.244076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.250182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.258682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 543/1093] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:56:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:56:58.261729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 551/1094] 10.222.0.48 () {56 vars in 1133 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:56:58.267235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.278319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.362923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.363898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003d53b-5eae-4972-81f7-cb1c15092be7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.375671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d7cb1-986d-4cc6-9ccc-d2e23fb290f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.396716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.397355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.397634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.397906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.409623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.410399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.410696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.411005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.421382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:56:58.428911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:58.430031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:56:58.431396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 552/1095] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:58.436216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.465765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.468586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.470301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.539389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50459ff-67df-4151-a954-a5071aac796d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.574313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 544/1096] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:56:58.574970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.576155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.576437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.576706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.577191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.577444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:58.577745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.579118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.592762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 553/1097] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:58.593799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.605693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.671886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.674985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8b9d7-a2f8-4c30-ba41-7e980d930f3b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.692173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372031e9-a915-42c1-8b21-153feef680b9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.708619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.709399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.709685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.710015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.720458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.721893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.725185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.726031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.726311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.726586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] PATH_INFO: `/v3/users/126ceafcef354edc8d835246b37ecb8b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.727775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] RBAC: Authorizing `identity:get_user(user_id=126ceafcef354edc8d835246b37ecb8b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.729165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.730776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 554/1098] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/users/126ceafcef354edc8d835246b37ecb8b => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:58.736430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:56:58.738865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:58.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:58.740749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:56:58.740749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:56:58.740749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:56:58.740749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:58.740749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 545/1099] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:58.745353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.811954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.833936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce15b8-0cfa-4a71-ab8e-b1b28565d9c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.842115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ad526-2295-4fa4-99b4-aed451c5005b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:58.867042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.867676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.867965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.868243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.874792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:58.875471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:58.875732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:58.876006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:58.876718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:58.877031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.878376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.889638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 546/1100] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:58.895075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:58.908954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:58.934249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:58.936203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:58.993043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7bf60-ecfe-4199-a730-4644a23feb89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:59.010587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:59.026632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:59.027441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:59.027729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:59.028006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:59.033715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 555/1101] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 06:56:59.034629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:59.036100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:59.039500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:56:59.050034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:56:59.051163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:56:59.052099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 547/1102] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:56:59.056761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:59.126072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:59.136091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f309a-27cf-4be2-970c-d122d1961622 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:59.156604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6914a-d686-447e-a28b-5c8830b4c32c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:59.169183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:59.170093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:59.170373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:59.170647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:59.171840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:59.173252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:59.174939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 556/1103] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:56:59 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:59.180839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:59.191460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:59.192325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:59.192610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:59.192887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:59.193672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:59.193981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:59.195478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:59.207462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 548/1104] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:56:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:59.213294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:56:59.234413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:59.286690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0309-a316-4be1-9fa6-10f07543f908 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:59.309719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741cce3-0b76-449e-b1d8-b1ce9b03dc35 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:56:59.321276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:59.321921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:59.322203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:59.322475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:59.323337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=cdc0d36b83844a979be5e2535b68274c, user.id=126ceafcef354edc8d835246b37ecb8b) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:59.323650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:59.325064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b9b46a-293a-452d-98b5-fa603ffd82dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:59.331284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 557/1105] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:59 2022] GET /identity/v3/role_assignments?user.id=126ceafcef354edc8d835246b37ecb8b&scope.project.id=cdc0d36b83844a979be5e2535b68274c&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:56:59.338550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:56:59.344949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:56:59.345622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:56:59.345906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:56:59.346182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:56:59.347102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] RBAC: Adding query filter params (role.id=bfdadde65b63412ca262dd804683eb22, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:56:59.347417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:56:59.348901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2daf2-c8cc-4c34-b69e-d50aabf73163 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:56:59.355646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 549/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:56:59 2022] GET /identity/v3/role_assignments?user.id=458dc4b52e3d4a13a8a4f801ddf2e6c4&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=bfdadde65b63412ca262dd804683eb22 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:57:01.390152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247604d-e690-4057-946f-bb48daa26915 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:01.390694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247604d-e690-4057-946f-bb48daa26915 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:01.390694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247604d-e690-4057-946f-bb48daa26915 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:01.392099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 558/1107] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:01.397097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:01.397509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:01.397651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:01.414339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:01.415539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:01.479258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 550/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:57:01.503918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247604d-e690-4057-946f-bb48daa26915 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:01.600753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247604d-e690-4057-946f-bb48daa26915 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:01.611020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:01.633726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:01.634496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:01.634793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:01.635071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:01.636214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:01.637633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b78ca3-7290-4af5-ab56-8b0e40ba3ffe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:01.639835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 559/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:57:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:01.647521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:01.741196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb875f-eabc-46c5-97f4-25ba57d088b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:01.765244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:01.774422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:01.775133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:01.775418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:01.775697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:01.776415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:01.777839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55977440-0699-44b8-8046-a14722eb5a9e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:01.797184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 551/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:57:01 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:57:03.846677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb1fc-12bd-448f-a51e-a8c5fa91b980 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:03.847344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb1fc-12bd-448f-a51e-a8c5fa91b980 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:03.847344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eb1fc-12bd-448f-a51e-a8c5fa91b980 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:03.848641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 560/1111] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:03.853946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:03.854393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:03.854491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:03.871319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:03.873078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:03.936877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 552/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:03.959812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eb1fc-12bd-448f-a51e-a8c5fa91b980 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:04.055894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eb1fc-12bd-448f-a51e-a8c5fa91b980 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:04.088170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:04.089008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:04.089293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:04.089570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] PATH_INFO: `/v3/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:04.091146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] RBAC: Authorizing `identity:get_role(role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:04.092500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:04.093826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 561/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:03 2022] GET /identity/v3/roles/1dfd6d1514104a199766879261c434f7 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:57:04.099356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:04.194310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31027a41-21a8-4a59-bc3c-5a0c683068de None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:04.206941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:04.228820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:04.229675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:04.229956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:04.230225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] PATH_INFO: `/v3/users/51f37ee68bcc46af9e53568e93ea1f3e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:04.243300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] RBAC: Authorizing `identity:get_user(user_id=51f37ee68bcc46af9e53568e93ea1f3e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:04.244723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:04.246334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 553/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:04 2022] GET /identity/v3/users/51f37ee68bcc46af9e53568e93ea1f3e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:04.252220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:04.297791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:04.348750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b9942-b560-4c39-9746-e9561691b5ff None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:04.381383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:04.382015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:04.382296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:04.382572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:04.446399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:04.447850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:04.490371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:04.541563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 562/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:04.547325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:04.643063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48f302-bce4-4f24-996c-de955dbc78f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:04.675514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:04.676354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:04.676635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:04.676908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:04.678169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:04.679643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c73460-af10-4f4f-ba14-3c1438554899 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:04.681354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 554/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:04 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:04.687609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:04.782166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b7696-aebb-4428-a2ff-84369414e3ba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:04.814592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:04.815253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:04.815535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:04.815810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:04.816618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=cdc0d36b83844a979be5e2535b68274c, user.id=51f37ee68bcc46af9e53568e93ea1f3e) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:04.816927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:04.818342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681b8e7-b49c-4059-af04-db34d30fb3e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:04.823512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 563/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:04 2022] GET /identity/v3/role_assignments?user.id=51f37ee68bcc46af9e53568e93ea1f3e&scope.project.id=cdc0d36b83844a979be5e2535b68274c&role.id=1dfd6d1514104a199766879261c434f7 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:06.773979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b840-21f0-4e9a-a191-20d557e6ffec None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:06.774529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b840-21f0-4e9a-a191-20d557e6ffec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:06.774529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8b840-21f0-4e9a-a191-20d557e6ffec None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:06.775930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 555/1118] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:06.781145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:06.781541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:06.781823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:06.798450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:06.799684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:06.802308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:06.863838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 564/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:06.886931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8b840-21f0-4e9a-a191-20d557e6ffec None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:06.926070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:06.983891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8b840-21f0-4e9a-a191-20d557e6ffec None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:07.016295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:07.017130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:07.017410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:07.017685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] PATH_INFO: `/v3/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:07.019159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] RBAC: Authorizing `identity:get_role(role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:07.020503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:07.021830 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 556/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:06 2022] GET /identity/v3/roles/1dfd6d1514104a199766879261c434f7 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:07.027596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:07.120459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089ac6-f6ca-4658-affa-2968a973b1d9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:07.152220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:07.153057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:07.153334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:07.154144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] PATH_INFO: `/v3/users/51f37ee68bcc46af9e53568e93ea1f3e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:07.155319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] RBAC: Authorizing `identity:get_user(user_id=51f37ee68bcc46af9e53568e93ea1f3e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:07.156729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:07.158301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 565/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:07 2022] GET /identity/v3/users/51f37ee68bcc46af9e53568e93ea1f3e => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:07.163862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:07.258647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230382c5-30f8-42f9-aa75-15532080d7ab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:07.265568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:07.291408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:07.292190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:07.292470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:07.292744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:07.293899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:07.295932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6022f0-5b52-4922-bce2-8a6dc3fc2f9f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:07.297589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 557/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:07 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:07.303750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:07.314421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:07.397996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48728692-e9ba-437c-8d25-7367433b319a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:07.430537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:07.431430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:07.431712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:07.431986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c/users/51f37ee68bcc46af9e53568e93ea1f3e/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:07.434408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc0d36b83844a979be5e2535b68274c, user_id=51f37ee68bcc46af9e53568e93ea1f3e, role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:07.435882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34250a0-d79b-4a92-a5dc-2d1a0538be05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:07.448028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 566/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:57:07 2022] PUT /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c/users/51f37ee68bcc46af9e53568e93ea1f3e/roles/1dfd6d1514104a199766879261c434f7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:57:09.427658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1feee-468a-4478-998f-644984e4a498 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:09.428227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1feee-468a-4478-998f-644984e4a498 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:09.428227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1feee-468a-4478-998f-644984e4a498 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:09.429572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 558/1124] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:09.435422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:09.435806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:09.436023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:09.452582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:09.453744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:09.511428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:09.517975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 567/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:09.541299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1feee-468a-4478-998f-644984e4a498 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:09.639608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1feee-468a-4478-998f-644984e4a498 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:09.670931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:09.671763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:09.672043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:09.672324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] PATH_INFO: `/v3/roles/1dfd6d1514104a199766879261c434f7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:09.673778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] RBAC: Authorizing `identity:get_role(role_id=1dfd6d1514104a199766879261c434f7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:09.675147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:09.676473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 559/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:09 2022] GET /identity/v3/roles/1dfd6d1514104a199766879261c434f7 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:09.681945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:09.777793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287ff10-7a76-4b2e-80d2-77d7847e60b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:09.810953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:09.811813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:09.812099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:09.812377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] PATH_INFO: `/v3/users/51f37ee68bcc46af9e53568e93ea1f3e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:09.813556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] RBAC: Authorizing `identity:get_user(user_id=51f37ee68bcc46af9e53568e93ea1f3e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:09.815004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:09.816610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 568/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:09 2022] GET /identity/v3/users/51f37ee68bcc46af9e53568e93ea1f3e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:09.822300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:09.863633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:09.916419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a1396-d596-4b59-be8f-dee7e4806478 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:09.949023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:09.949648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:09.949929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:09.950201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:10.013854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:10.015326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:10.054262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:10.109893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 560/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:10.115654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:10.178846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:10.212536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b824-a576-4395-a8d7-d9daccd00c99 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:10.254148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:10.255019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:10.255392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:10.259745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] PATH_INFO: `/v3/projects/cdc0d36b83844a979be5e2535b68274c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:10.260920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] RBAC: Authorizing `identity:get_project(project_id=cdc0d36b83844a979be5e2535b68274c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:10.262340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3184d-e99d-4a56-8ddf-bf4bcef82844 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:10.265687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 569/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:10 2022] GET /identity/v3/projects/cdc0d36b83844a979be5e2535b68274c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:10.271928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:10.367086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdb65f-c5d9-4225-968e-a911695a8132 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:10.399227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:10.399871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:10.400156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:10.400433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:10.401256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] RBAC: Adding query filter params (role.id=1dfd6d1514104a199766879261c434f7, scope.project.id=cdc0d36b83844a979be5e2535b68274c, user.id=51f37ee68bcc46af9e53568e93ea1f3e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:10.401568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:10.403015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bc0fe-1ccf-44aa-ba21-7920fa318eb4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:10.409117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 561/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:10 2022] GET /identity/v3/role_assignments?user.id=51f37ee68bcc46af9e53568e93ea1f3e&scope.project.id=cdc0d36b83844a979be5e2535b68274c&role.id=1dfd6d1514104a199766879261c434f7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:12.434333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb672c1-1040-480f-a981-d7bdc34ae0a3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:12.434867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb672c1-1040-480f-a981-d7bdc34ae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:12.435036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb672c1-1040-480f-a981-d7bdc34ae0a3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:12.436405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 570/1131] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:12.441549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:12.441977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:12.442106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:12.453037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:12.458724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:12.459927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:12.523250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 562/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:12.547538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb672c1-1040-480f-a981-d7bdc34ae0a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:12.643708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb672c1-1040-480f-a981-d7bdc34ae0a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:12.676350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:12.677116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:12.677399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:12.677674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:12.678856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:12.680267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2ca6-4b92-454e-ac3d-1c67e239b508 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:12.681829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 571/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:57:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:12.687641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:12.781925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693a35c-7f7a-4769-a076-ec1a58479259 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:12.813961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:12.814606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:12.814922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:12.815198 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:12.815906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:12.817316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a46147-55fe-4ac5-a42c-3e22355b943b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:12.821220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:12.843957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 563/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:57:12 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:57:14.875369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc57192-ad02-4977-9255-2c1939a77515 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:14.875903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc57192-ad02-4977-9255-2c1939a77515 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:14.875903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc57192-ad02-4977-9255-2c1939a77515 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:14.877274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 572/1135] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:14.882280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:14.882726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:14.882854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:14.899626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:14.900791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:14.966260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 564/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:14.991469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc57192-ad02-4977-9255-2c1939a77515 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:14.994324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:15.090763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc57192-ad02-4977-9255-2c1939a77515 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:15.123776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:15.124549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:15.124832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:15.125110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:15.126311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:15.127712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf7fec1-41fb-4bf4-b124-4102073aed12 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:15.129290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 573/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:57:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:15.135028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:15.230040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3ebe8-740d-4329-84b5-5e5dbd646fc6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:15.246688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:15.264333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:15.264988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:15.265268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:15.265649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:15.266370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:15.267811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8307199-3a4d-4129-a5cc-bbd48ab57f2f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:15.286652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 565/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:57:15 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:57:17.331133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc8e4-8a40-4ac6-92eb-a260f5463751 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:17.331661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc8e4-8a40-4ac6-92eb-a260f5463751 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:17.331661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bc8e4-8a40-4ac6-92eb-a260f5463751 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:17.333043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 574/1139] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:17.338181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:17.338570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:17.338739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:17.355548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:17.356723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:17.420040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 566/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:17.443119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bc8e4-8a40-4ac6-92eb-a260f5463751 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:17.510319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:17.540770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bc8e4-8a40-4ac6-92eb-a260f5463751 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:17.572513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:17.573279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:17.573559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:17.573829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:17.583982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:17.592038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:17.593186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:17.594284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 575/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:17.599069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:17.691866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46dc25-30cd-4867-abdc-233e2ca004b6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:17.694035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:17.725738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:17.726388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:17.726669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:17.726973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:17.727748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:17.728059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:17.729409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:17.738795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 567/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:17.744493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:17.840156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a50e9-9a82-4b10-b07d-a42d361cddd2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:17.868187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:17.874336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:17.875224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:17.875504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:17.875775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] PATH_INFO: `/v3/users/17d633d743374cf394460fe5d25a10c9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:17.889020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] RBAC: Authorizing `identity:get_user(user_id=17d633d743374cf394460fe5d25a10c9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:17.890440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:17.892073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 576/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:17 2022] GET /identity/v3/users/17d633d743374cf394460fe5d25a10c9 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:17.897924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:17.994663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4edcda-4cd1-44b8-885c-1642d92bc1de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:18.026950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:18.027613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:18.027898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:18.028177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:18.060885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:18.093977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:18.095379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:18.189994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 568/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:18.195792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:18.293598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a6422-7ed2-4fb7-95b1-261eb6c016ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:18.327446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:18.328286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:18.328565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:18.328836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] PATH_INFO: `/v3/projects/4412bb5fbdec4b94a8427799f639c187` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:18.329987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] RBAC: Authorizing `identity:get_project(project_id=4412bb5fbdec4b94a8427799f639c187)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:18.331444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770ba38-0af0-49ec-b741-da18df430fef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:18.333102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 577/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:18 2022] GET /identity/v3/projects/4412bb5fbdec4b94a8427799f639c187 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:18.339204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:18.402501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:18.437035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212fee2-5223-4135-a541-c7a759e547b1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:18.469524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:18.470159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:18.470444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:18.470723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:18.471601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=4412bb5fbdec4b94a8427799f639c187, user.id=17d633d743374cf394460fe5d25a10c9) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:18.471913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:18.473330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17933b-8e0a-4019-93ac-e3d7ced38304 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:18.478567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 569/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:18 2022] GET /identity/v3/role_assignments?user.id=17d633d743374cf394460fe5d25a10c9&scope.project.id=4412bb5fbdec4b94a8427799f639c187&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:57:20.450784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ca9a-34e6-4d06-b3fd-b058d431c31d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:20.451331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ca9a-34e6-4d06-b3fd-b058d431c31d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:20.451428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ca9a-34e6-4d06-b3fd-b058d431c31d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:20.452738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 578/1147] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:20.457773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:20.458176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:20.458321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:20.474540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:20.475756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:20.539047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 570/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:20.562538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80ca9a-34e6-4d06-b3fd-b058d431c31d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:20.616961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:20.658665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80ca9a-34e6-4d06-b3fd-b058d431c31d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:20.691068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:20.691839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:20.692123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:20.692398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:20.702174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:20.709678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:20.710807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:20.711942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 579/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:20.717237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:20.773453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:20.812990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1452b-7f6d-4d38-a0c9-e060d4721d62 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:20.845371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:20.846013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:20.846292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:20.846561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:20.847317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:20.847618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:20.848989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:20.856681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 571/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:20.862991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:20.959659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0670d4-b95b-4ab6-a746-a4885d0173ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:20.975880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:20.993903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:20.994761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:20.995074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:20.995351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] PATH_INFO: `/v3/users/17d633d743374cf394460fe5d25a10c9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:20.996528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=17d633d743374cf394460fe5d25a10c9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:20.997978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:20.999618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 580/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:20 2022] GET /identity/v3/users/17d633d743374cf394460fe5d25a10c9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:21.005503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:21.107092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcacba-e50c-4b4d-9bee-621b9917d7f1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:21.128873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:21.140948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:21.141798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:21.142082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:21.142357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] PATH_INFO: `/v3/projects/4412bb5fbdec4b94a8427799f639c187` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:21.143552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] RBAC: Authorizing `identity:get_project(project_id=4412bb5fbdec4b94a8427799f639c187)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:21.144974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b06483-3901-46d2-8707-8bdd21bade70 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:21.146641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 572/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:21 2022] GET /identity/v3/projects/4412bb5fbdec4b94a8427799f639c187 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:21.152538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:21.248758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab429-eea4-414d-9685-8ab711a8f7a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:21.280854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:21.281717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:21.281996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:21.282267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] PATH_INFO: `/v3/projects/4412bb5fbdec4b94a8427799f639c187/users/17d633d743374cf394460fe5d25a10c9/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:21.284239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] RBAC: Authorizing `identity:create_grant(project_id=4412bb5fbdec4b94a8427799f639c187, user_id=17d633d743374cf394460fe5d25a10c9, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:21.285646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a4ab0-3ba6-49f5-bfb9-641aa2ae152d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:21.298682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 581/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:57:21 2022] PUT /identity/v3/projects/4412bb5fbdec4b94a8427799f639c187/users/17d633d743374cf394460fe5d25a10c9/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:57:23.291662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038da1-06d2-40ef-9f87-36c9e103cace None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.292124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038da1-06d2-40ef-9f87-36c9e103cace None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.292212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038da1-06d2-40ef-9f87-36c9e103cace None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:23.293550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 573/1154] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:23.299145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.299516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.299717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:23.317126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:23.318312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:23.356401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:23.383156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 582/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:23.406402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8038da1-06d2-40ef-9f87-36c9e103cace None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:23.503870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8038da1-06d2-40ef-9f87-36c9e103cace None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:23.535297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:23.536064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.536344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.536619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:23.546827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:23.554388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:23.555574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:57:23.556449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 574/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:23.563677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:23.659606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293bbc1-1a9b-439f-869c-56832d9447ef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:23.692607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:23.693259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.693546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.693826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:23.694569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:23.694910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:23.696273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:23.704049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 583/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:23.709664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:23.761890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:23.810733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899849c5-46ac-43e2-9d84-a6688a06650e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:23.845773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:23.846647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.846962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.849325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] PATH_INFO: `/v3/users/17d633d743374cf394460fe5d25a10c9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:23.850555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] RBAC: Authorizing `identity:get_user(user_id=17d633d743374cf394460fe5d25a10c9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:23.852020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:23.855801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 575/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:23 2022] GET /identity/v3/users/17d633d743374cf394460fe5d25a10c9 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:23.862256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:23.895778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:23.959613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430467b-d7a0-4504-8079-5ca602f42e76 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:23.992197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:23.992833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:23.993118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:23.993393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:24.056912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:24.058305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:24.090175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:24.151736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 584/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:24.158313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:24.276391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a71f-4bb4-465c-9430-939feff9e958 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:24.292768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:24.309702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:24.310522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:24.310827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:24.311108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] PATH_INFO: `/v3/projects/4412bb5fbdec4b94a8427799f639c187` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:24.312278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=4412bb5fbdec4b94a8427799f639c187)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:24.313703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc28665-061e-4c4c-a8c7-5fdbf5ece8d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:24.315419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 576/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:24 2022] GET /identity/v3/projects/4412bb5fbdec4b94a8427799f639c187 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:24.321561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:24.418557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7c40f-1ae3-43c3-9ae4-b54aab985e73 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:24.451967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:24.452605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:24.452883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:24.453156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:24.453970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=4412bb5fbdec4b94a8427799f639c187, user.id=17d633d743374cf394460fe5d25a10c9) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:24.454278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:24.455717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9344896-c1b0-492e-b3c6-5794f4e4c35a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:24.458466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:24.461788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 585/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:24 2022] GET /identity/v3/role_assignments?user.id=17d633d743374cf394460fe5d25a10c9&scope.project.id=4412bb5fbdec4b94a8427799f639c187&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:26.463671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58891275-6b59-47a7-93b1-dfb68f74c03e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:26.464190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58891275-6b59-47a7-93b1-dfb68f74c03e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:26.464190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58891275-6b59-47a7-93b1-dfb68f74c03e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:26.465494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 577/1162] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:26.470759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:26.471179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:26.471342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:26.487896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:26.489069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:26.554317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 586/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:26.577627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58891275-6b59-47a7-93b1-dfb68f74c03e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:26.676638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58891275-6b59-47a7-93b1-dfb68f74c03e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:26.713603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:26.714290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:26.714519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:26.714815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:26.765981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:26.778813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:26.780775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:26.876820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 578/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:26.882660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:26.975922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4be8e-03ea-460d-a427-9f26017a2cfe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:27.003570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:27.009446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:27.010243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:27.010529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:27.010833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:27.017850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:27.019317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c1b0f927-9f6a-4128-885a-da76956ec136 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:27.033675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:27.034903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:27.035810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 587/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:57:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:27.040742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:27.121503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:27.136126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec1924-3aef-42b0-ae12-6c59b1ebd8ec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:27.169888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:27.170530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:27.170832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:27.171109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:27.171817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:27.172122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:27.173467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6df7-0e1f-4765-83a7-0d5dcaccc0bc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:27.180300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 579/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Fri Jul 1 06:57:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:29.173537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83eb18-28bf-40a3-a3f4-c6214455caa9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:29.174066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83eb18-28bf-40a3-a3f4-c6214455caa9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:29.174066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83eb18-28bf-40a3-a3f4-c6214455caa9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:29.175487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 588/1167] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:29.181446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb449-12b9-4534-acef-e13a8c0e6474 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:29.181846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb449-12b9-4534-acef-e13a8c0e6474 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:29.181999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bb449-12b9-4534-acef-e13a8c0e6474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:29.199161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9f1bb449-12b9-4534-acef-e13a8c0e6474 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:29.200345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9f1bb449-12b9-4534-acef-e13a8c0e6474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:29.264702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 580/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:29.287290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83eb18-28bf-40a3-a3f4-c6214455caa9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:29.386290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83eb18-28bf-40a3-a3f4-c6214455caa9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:29.418426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:29.419089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:29.419374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:29.420183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:29.420870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:29.422243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa51f5-2a2d-4085-9727-ff1ab3c4019f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:29.437034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 589/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Fri Jul 1 06:57:29 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:57:31.447155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefacf5-bf96-4103-91db-bc7622e983c6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:31.447665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefacf5-bf96-4103-91db-bc7622e983c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:31.447749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefacf5-bf96-4103-91db-bc7622e983c6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:31.449094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 581/1170] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:31.454656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:31.455045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:31.455251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:31.471374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:31.472492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:31.516356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:31.537288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 590/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:31.564894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefacf5-bf96-4103-91db-bc7622e983c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:31.642929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:31.662290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefacf5-bf96-4103-91db-bc7622e983c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:31.695466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:31.696307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:31.696586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:31.696858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] PATH_INFO: `/v3/domains/7102922de5e54a58abc8e6cc9a3b22cf` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:31.706836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7102922de5e54a58abc8e6cc9a3b22cf)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:31.708256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1679d-aa25-4876-89b4-d2a09591af63 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:31.709837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 582/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Fri Jul 1 06:57:31 2022] GET /identity/v3/domains/7102922de5e54a58abc8e6cc9a3b22cf => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:31.715602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:31.812628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e08134-b9cd-4484-b1f4-c0e98efb7f1e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:31.844935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:31.845582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:31.845871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:31.846151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:31.846874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:31.848301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3030b-01ba-4cc1-b9e4-72137c87e381 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:31.875828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 591/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:57:31 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:57:33.933511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c28b-0050-4ee1-8c9c-b3cc390701b9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:33.934304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c28b-0050-4ee1-8c9c-b3cc390701b9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:33.934411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c28b-0050-4ee1-8c9c-b3cc390701b9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:33.935768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 583/1174] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:33.941159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:33.941558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:33.941717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:33.958797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:33.960002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:33.985502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:34.025504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 592/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:34.051153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996c28b-0050-4ee1-8c9c-b3cc390701b9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:34.150177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996c28b-0050-4ee1-8c9c-b3cc390701b9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:34.182979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:34.184132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:34.184363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:34.184639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] PATH_INFO: `/v3/domains/7102922de5e54a58abc8e6cc9a3b22cf` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:34.186392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7102922de5e54a58abc8e6cc9a3b22cf)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:34.187846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71f7de-4474-48ba-af82-dcf13b771217 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:34.189456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 584/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Fri Jul 1 06:57:34 2022] GET /identity/v3/domains/7102922de5e54a58abc8e6cc9a3b22cf => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:34.195389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:34.295705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea748-6ff7-439d-85b3-c5bb71db51f7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:34.328637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:34.329295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:34.329578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:34.329855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:34.330551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:34.332147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92bf1f-4c08-4e36-80ab-deb54f6372f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:34.352125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 593/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:57:34 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:57:36.381580 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f945-3d19-4871-a249-f674ab710af6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:36.382128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f945-3d19-4871-a249-f674ab710af6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:36.382128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37f945-3d19-4871-a249-f674ab710af6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:36.383495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 585/1178] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:36.389262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:36.389648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:36.389816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:36.405998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:36.407215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:36.419841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:36.472850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 594/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:36.495581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37f945-3d19-4871-a249-f674ab710af6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:36.506564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:36.594693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37f945-3d19-4871-a249-f674ab710af6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:36.626697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:36.627492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:36.627774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:36.628049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:36.637640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:36.645622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:36.646802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:57:36.648014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 586/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:36.652723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:36.746031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf4b16-df27-4849-a4e8-7f92162c26e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:36.778359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:36.779055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:36.779336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:36.779608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:36.780337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:36.780640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:36.781985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:36.789592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 595/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:36.795651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:36.859474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:36.892063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466700b-e077-4bf8-83e6-050a7ee1fb17 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:36.923804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:36.924652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:36.924931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:36.925204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] PATH_INFO: `/v3/users/f3d4adb5e33546b9ad3664ce56a42998` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:36.938506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] RBAC: Authorizing `identity:get_user(user_id=f3d4adb5e33546b9ad3664ce56a42998)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:36.939960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:36.941567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 587/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:36 2022] GET /identity/v3/users/f3d4adb5e33546b9ad3664ce56a42998 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:36.947855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:36.960274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:37.042849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3259f8-e49a-4ff6-a01b-b799a28d1b10 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:37.075048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:37.075680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:37.075990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:37.076263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:37.141346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:37.142795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:37.153226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:37.237540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 596/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 06:57:37.243368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:37.339076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270ff7-6cda-49e8-9493-0436b720ba2e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:37.363783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:37.372444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:37.373287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:37.373563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:37.373832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] PATH_INFO: `/v3/projects/9c5352cdd0144343911740ae3f70b9a0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:37.375070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] RBAC: Authorizing `identity:get_project(project_id=9c5352cdd0144343911740ae3f70b9a0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:37.376494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c2440-5607-43e2-895d-3caebb3fa737 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:37.378146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 588/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:37 2022] GET /identity/v3/projects/9c5352cdd0144343911740ae3f70b9a0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:37.384197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:37.478134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6a547-b361-400d-b91e-91821af8e2c5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:37.488793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:37.511340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:37.511972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:37.512252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:37.512523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:37.513879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=9c5352cdd0144343911740ae3f70b9a0, user.id=f3d4adb5e33546b9ad3664ce56a42998) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:37.514185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:37.515628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188462bf-ccd9-4991-9e84-c314f37bc6b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:37.520858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 597/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:37 2022] GET /identity/v3/role_assignments?user.id=f3d4adb5e33546b9ad3664ce56a42998&scope.project.id=9c5352cdd0144343911740ae3f70b9a0&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:39.499627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86b654-e90e-4c9f-828a-025b218f7734 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:39.500132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86b654-e90e-4c9f-828a-025b218f7734 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:39.500132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86b654-e90e-4c9f-828a-025b218f7734 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:39.501435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 589/1186] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:39.506696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:39.507125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:39.507280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:39.524238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:39.525428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:39.592354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 598/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:39.615307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86b654-e90e-4c9f-828a-025b218f7734 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:39.712779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86b654-e90e-4c9f-828a-025b218f7734 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:39.744932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:39.745701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:39.745983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:39.746259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:39.756601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:39.764175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:39.765259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:57:39.766167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 590/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:57:39.771421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:39.866338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1e8b4-ede7-4e75-b3cb-9edf17b958ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:39.868583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:39.900070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:39.900730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:39.901014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:39.901290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:39.902038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:39.902349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:39.903785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:39.911665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 599/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:57:39.917526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:39.970978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:40.014478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cbdc3-cbf2-47e6-8810-a65f816c7d03 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:40.045955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:40.046817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:40.047101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:40.047373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] PATH_INFO: `/v3/users/f3d4adb5e33546b9ad3664ce56a42998` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:40.048538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d4adb5e33546b9ad3664ce56a42998)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:40.049947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:40.051547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 591/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:39 2022] GET /identity/v3/users/f3d4adb5e33546b9ad3664ce56a42998 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:40.057234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:40.151827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f7d2-ecb7-4ca7-949a-ab5545046def None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:40.183653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:40.184734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:40.186588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:40.186588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] PATH_INFO: `/v3/projects/9c5352cdd0144343911740ae3f70b9a0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:40.186588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] RBAC: Authorizing `identity:get_project(project_id=9c5352cdd0144343911740ae3f70b9a0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:40.188020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdf86d-b3bc-4453-a941-7d68a8287454 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:40.189717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 600/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:40 2022] GET /identity/v3/projects/9c5352cdd0144343911740ae3f70b9a0 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:57:40.195642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:40.293672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5e9a1-4f0a-4c84-9870-6afa5170d4b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:40.315361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:40.326951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:40.327834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:40.328120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:40.328397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] PATH_INFO: `/v3/projects/9c5352cdd0144343911740ae3f70b9a0/users/f3d4adb5e33546b9ad3664ce56a42998/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:40.330358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c5352cdd0144343911740ae3f70b9a0, user_id=f3d4adb5e33546b9ad3664ce56a42998, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:40.331821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ddcf1-f22d-4a56-b5d1-cd3e4cab4354 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:40.344209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 592/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:57:40 2022] PUT /identity/v3/projects/9c5352cdd0144343911740ae3f70b9a0/users/f3d4adb5e33546b9ad3664ce56a42998/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:57:42.325074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41455602-b197-4a3f-b791-e8395742dfe4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:42.326099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41455602-b197-4a3f-b791-e8395742dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:42.326192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41455602-b197-4a3f-b791-e8395742dfe4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:42.327564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 601/1193] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:57:42.332582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:42.332962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:42.333133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:42.350010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:57:42.351212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:57:42.414250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 593/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:57:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:57:42.437716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41455602-b197-4a3f-b791-e8395742dfe4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:42.474691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:42.538562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41455602-b197-4a3f-b791-e8395742dfe4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:42.570806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:42.571579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:42.571861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:42.572133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:42.582375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:57:42.589867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:57:42.590987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:57:42.592172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 602/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:57:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:57:42.596941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:42.691394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cbcbe-0f25-4f89-9689-dd101cd7956d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:42.704388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:42.725369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:42.725969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:42.726250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:42.726530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:42.727300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:42.727608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:42.728956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:42.737351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 594/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:42.743012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:42.832808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:42.839861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa496-3999-4cb3-a51f-67c5469061c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:42.871636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:42.873042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:42.873342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:42.873621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] PATH_INFO: `/v3/users/f3d4adb5e33546b9ad3664ce56a42998` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:42.874834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d4adb5e33546b9ad3664ce56a42998)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:42.876256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:42.877845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 603/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Fri Jul 1 06:57:42 2022] GET /identity/v3/users/f3d4adb5e33546b9ad3664ce56a42998 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:42.883421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:42.978540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f836fa-f363-4fe0-82ef-a797d6ad4b83 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:43.010952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:43.011583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:43.011868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:43.012145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:43.051947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:43.076653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:43.078077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:43.171494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 595/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:57:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:57:43.177240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:43.273038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37887fd8-0eca-43b1-8e0a-236afa507cd4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:43.306179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:43.307044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:43.307325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:43.307598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] PATH_INFO: `/v3/projects/9c5352cdd0144343911740ae3f70b9a0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:43.308744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=9c5352cdd0144343911740ae3f70b9a0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:43.310457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126428e-7c0d-4349-ab2e-51406e202eb4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:43.312151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 604/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Fri Jul 1 06:57:43 2022] GET /identity/v3/projects/9c5352cdd0144343911740ae3f70b9a0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:43.318329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:57:43.389254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:43.413402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1417-020a-4f3e-9238-c04b95cf8961 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:57:43.446737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:57:43.447399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:57:43.447759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:57:43.448035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:57:43.448875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=9c5352cdd0144343911740ae3f70b9a0, user.id=f3d4adb5e33546b9ad3664ce56a42998) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:57:43.449181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:57:43.450586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce2bc8-8554-4060-87ab-1d631fd3e03d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:57:43.456150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 596/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:57:43 2022] GET /identity/v3/role_assignments?user.id=f3d4adb5e33546b9ad3664ce56a42998&scope.project.id=9c5352cdd0144343911740ae3f70b9a0&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:57:53.302201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:57:53.453582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=31 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:38.431316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679241-8e7f-4828-9c93-c51e91314693 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:38.431316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679241-8e7f-4828-9c93-c51e91314693 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:38.431316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679241-8e7f-4828-9c93-c51e91314693 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:38.433470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 605/1201] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:38.440616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:38.441519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:38.442022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:38.464787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:38.466143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:38.548483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 597/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:38.575782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa679241-8e7f-4828-9c93-c51e91314693 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:38.716948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa679241-8e7f-4828-9c93-c51e91314693 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:38.763206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:38.764016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:38.764298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:38.764573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:38.777841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:38.780101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:38.805156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:38.806228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:38.807286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:38.807286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:38.807286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:38.807286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:38.807286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 606/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:38.815680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:38.955887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326e0b7-f2ba-4aa3-84c8-54a597a86c0c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:38.990606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:38.999122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:38.999832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:39.000110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:39.000364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:39.001085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:39.001391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:39.002750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac2094-209e-4634-a8f7-d38376cc7eea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:39.021298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 598/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:39.025371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:39.100870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:39.165553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad1dc2-2a63-4c53-b83d-05bd12b4775a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:39.217574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:39.218476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:39.219041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:39.219537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:39.220502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:39.222110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118cfb6-18be-43b4-9711-5f99f961f628 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:39.250879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 607/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Fri Jul 1 06:58:39 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 06:58:41.520760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90a70-93b3-452c-b278-a93a505a9a8d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:41.521286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90a70-93b3-452c-b278-a93a505a9a8d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:41.521286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90a70-93b3-452c-b278-a93a505a9a8d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:41.522682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 599/1206] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:41.528170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:41.528609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:41.528714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:41.548249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:41.549704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:41.625079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 608/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:41.649988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c90a70-93b3-452c-b278-a93a505a9a8d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:41.771595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c90a70-93b3-452c-b278-a93a505a9a8d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:41.813684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:41.814555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:41.814871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:41.815148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:41.826228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:58:41.833969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:41.835222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:41.836099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:41.843043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 600/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:58:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:41.844815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:41.889803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:41.962211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e2201-ba7e-4b17-853e-acbb6db6aad4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.002507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.003215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.003506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.003787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:42.004543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:42.004860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:42.006229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:42.018984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 609/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:58:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:42.026894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:42.094071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:42.156186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ab6f2-7175-4cd2-94f6-431a69057cbd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.196499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.197365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.197650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.197977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:42.213893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:42.215500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:42.234512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:42.235576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:42.236483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:42.236483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:42.236483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:42.236483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:42.236483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 601/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:42.242142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:42.357596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:42.362566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13387985-93c7-4224-86b9-798f17c7e8a3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.405088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.405690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.405976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.406260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:42.407033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:42.407391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:42.408734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:42.422059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 610/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:42.430052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:42.549674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bcd58e-a4a9-491a-81f0-0cc885f15394 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.552315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:42.590993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.591832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.592115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.592390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:42.600843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:42.602453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:58:42.617847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:42.619041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:42.620216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:42.623238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 602/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:42.624611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:42.739802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ea672-cf1d-4dd0-90d4-adaafb561efe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.782839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.783906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.784342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.784771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:42.785931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:42.786297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:42.787827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:42.809079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 611/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:58:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:42.815991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:42.954218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e9f16-93d1-4426-9cfc-bcb6a5e5d8fa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:42.996321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:42.996971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:42.997330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:42.998219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:43.031430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:43.080149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:43.081736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:43.204676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 603/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:58:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 01 06:58:43.205867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:43.234573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:43.328863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a71f1-2d7d-43d3-916c-052a7337d6ad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:43.374650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:43.375474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:43.375759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:43.376035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:43.387948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:43.389695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:43.409098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:43.411284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:43.412197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:43.412197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:43.412197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:43.412197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:43.412197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 612/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:58:43.417009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:43.486750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:43.532046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba29f6f-8aef-4fdd-9f57-93edc009f710 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:43.573862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:43.576135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:43.576626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:43.577100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:43.578206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:43.578737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:43.580968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:43.596352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 604/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:43.602363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:43.683147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:43.718318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb1606-f3ac-4c18-887a-0e8741350b05 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:43.754752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:43.755489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:43.755767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:43.756038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:43.763772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:43.765543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:58:43.783212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:43.784391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:43.785551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 613/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:43.792903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:43.905459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9764e-821e-4267-8e6d-0f0fb95f0790 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:43.931712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:43.945837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:43.946501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:43.947030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:43.947364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:43.948169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:43.948584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:43.950315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f559c9-654d-4319-a73f-25316eb3b348 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:43.965881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 605/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:58:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:43.972728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:44.093501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821c12c-65e3-42cb-b05c-dd0118508c9e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:44.123816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:44.130830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:44.131489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:44.131771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:44.132046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:44.132875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=41b5461ab9d746b2b45cbc1960dececd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:44.133191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:44.134604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc8bda6-3043-466a-b16e-e465d3d04717 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:44.141633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 614/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:58:43 2022] GET /identity/v3/role_assignments?user.id=41b5461ab9d746b2b45cbc1960dececd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:46.360253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327f53-44a6-4dc3-825d-36717aba0faf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:46.360810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327f53-44a6-4dc3-825d-36717aba0faf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:46.360810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14327f53-44a6-4dc3-825d-36717aba0faf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:46.362191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 606/1220] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:46.367694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:46.368097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:46.368327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:46.386991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:46.388358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:46.462650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 615/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:46.485265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14327f53-44a6-4dc3-825d-36717aba0faf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:46.599387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14327f53-44a6-4dc3-825d-36717aba0faf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:46.633253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:46.634038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:46.634347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:46.634591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:46.645396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:58:46.652990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:46.654662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:46.655539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 607/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:58:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:46.661062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:46.758517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e0ae8-3c37-4e36-a537-07fb62c45e14 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:46.781946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:46.794460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:46.795593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:46.795896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:46.796287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:46.797170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:46.797704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:46.799471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:46.809626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 616/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:58:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:46.816079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:46.861556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:46.924493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484c61c-d4e9-4311-a095-9ba41ebc29bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:46.963682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:46.964511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:46.964835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:46.965119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:46.976650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:46.978078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:46.995691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:46.996726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:46.998199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:46.998199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:46.998199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:46.998199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:46.998199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 608/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:47.003526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.117461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dba509-2faa-418d-84da-7aa748e6e4c8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:47.151854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:47.152514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:47.152798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:47.153074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:47.153797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:47.154103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:47.155513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:47.167834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 617/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:47.174246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.279527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:47.287643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385c57-794a-4c23-a686-fe3309829ed2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:47.325020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:47.325843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:47.326124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:47.326418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:47.334164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:47.335913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:58:47.352135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:47.353177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:47.354138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 609/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:47.360395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.415250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:47.470333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26ffc7-d9dc-42c7-9f04-5cbec44727c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:47.507219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:47.507884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:47.508164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:47.508432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:47.509260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:47.509565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:47.511037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:47.529472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 618/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:47.535441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.640778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f712-edaf-411a-b703-7ba7582e06b6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:47.678303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:47.679153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:47.679438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:47.679741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:47.692152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:47.693582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:47.710298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:47.712463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:47.714215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:47.714215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:47.714215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:47.714215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:47.714215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 610/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:47.718872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.829944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33886e-bd3b-4ecd-84e1-fd1c821a3ba3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:47.832354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:47.867283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:47.867944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:47.868332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:47.868493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:47.869226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:47.869521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:47.870913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:47.882253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 619/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:58:47.888928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:47.898570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:48.005434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7f384-48ce-46af-9649-b2867575e1a3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:48.040529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:48.041429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:48.041776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:48.042082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:48.049211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:48.050690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:58:48.065002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:48.066078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:48.067032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 611/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:48.072129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:48.181263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869372d-605a-414e-9aed-e704852e1fa2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:48.217504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:48.218235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:48.218526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:48.218815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:48.219602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:48.219907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:48.221335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03267a27-1ae9-4869-86ea-ed2b63bbf4cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:48.235240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 620/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:48.241818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:48.315563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:48.354106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f25b1-3fe4-4910-ba3f-61638f1b92e1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:48.391839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:48.392896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:48.393189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:48.393472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/41b5461ab9d746b2b45cbc1960dececd/roles/8e6112fb4a374ad58c20fbb3e90b17d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:48.409228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=41b5461ab9d746b2b45cbc1960dececd, role_id=8e6112fb4a374ad58c20fbb3e90b17d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:48.410720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9b543-08da-4206-bd55-48309e6a807c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:48.423642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 612/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:58:48 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/41b5461ab9d746b2b45cbc1960dececd/roles/8e6112fb4a374ad58c20fbb3e90b17d8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:58:50.521811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9c77-f6a9-4987-8773-d5b058d85c73 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:50.522345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9c77-f6a9-4987-8773-d5b058d85c73 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:50.522443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9c77-f6a9-4987-8773-d5b058d85c73 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:50.524010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 621/1233] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:50.529857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:50.530316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:50.530423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:50.547358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:50.548568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:50.615871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 613/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:50.640487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd9c77-f6a9-4987-8773-d5b058d85c73 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:50.667524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:50.746684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd9c77-f6a9-4987-8773-d5b058d85c73 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:50.785118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:50.786095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:50.786365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:50.786648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:50.798819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:58:50.808130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:50.809483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 01 06:58:50.811058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:50.812152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 622/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Fri Jul 1 06:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:50.817685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:50.870342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:50.932702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842146b-b38f-44d0-a1ac-abfff7922ee8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:50.974130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:50.975011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:50.975357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:50.975678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:50.976567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:50.976974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:50.978669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:50.988324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 614/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Fri Jul 1 06:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:50.994138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:51.085571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:51.104969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a70d6-7c5e-4f12-9b81-ed26233ffb40 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:51.143814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:51.144587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:51.144866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:51.145138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:51.157378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:51.158931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:51.175993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:51.177005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:51.178074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:51.178074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:51.178074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:51.178074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:51.178074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 623/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:51.184760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:51.297208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb586afa-e7ff-4012-b92a-4aa166eeab8f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:51.306574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:51.335838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:51.336416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:51.336721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:51.337005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:51.337831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:51.338144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:51.339714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:51.352564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 615/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:51.358250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:51.472863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e717-3f10-45b3-86f9-00b379545e8f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:51.506729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:51.507543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:51.507836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:51.508099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:51.510998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:51.516776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:51.518349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:58:51.532387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:51.534243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:51.535634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 624/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:51.539488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:51.645642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8bdc7-725d-4568-aeaf-559e6fe9f223 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:51.683931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:51.685536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:51.685819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:51.686090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:51.686955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:51.687358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:51.688975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:51.707514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 616/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:58:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:58:51.713995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:51.819681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5aad1-9b79-4702-a001-fec20ff1ce85 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:51.855451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:51.856099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:51.856379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:51.857170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:51.922157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:51.930046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:51.931924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.048343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 625/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:58:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:58:52.054125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:52.083682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:52.167754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356110ca-d9d4-4481-a48d-0c10722619ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:52.208420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395139b-8736-4251-af5f-90077106a556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:52.209116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395139b-8736-4251-af5f-90077106a556 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:52.209398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395139b-8736-4251-af5f-90077106a556 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:52.209672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395139b-8736-4251-af5f-90077106a556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:52.221108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395139b-8736-4251-af5f-90077106a556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:52.222586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395139b-8736-4251-af5f-90077106a556 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f395139b-8736-4251-af5f-90077106a556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:52.241168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:52.242295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:52.243403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:52.243403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:52.243403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:52.243403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:52.243403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 617/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:52.248105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:52.348422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:52.354812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeea332-d27a-488d-97d3-f236ec4dca9b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:52.389988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:52.390748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:52.391094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:52.391391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:52.392193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:52.392531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:52.394052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.406989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 626/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:52.412857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395139b-8736-4251-af5f-90077106a556 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:52.497120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:52.520828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395139b-8736-4251-af5f-90077106a556 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:52.555664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:52.556452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:52.556730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:52.557002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:52.564029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:52.565464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:58:52.579450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:52.580462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:52.581409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:52.583633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 618/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:58:52.586507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:52.691052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faa6f1-7892-4874-bbac-4bccd5a53fef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:52.728676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:52.729393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:52.729679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:52.729956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:52.730738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:52.731084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:52.732541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500647a-f4a0-4364-9de3-722cdc763b7e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.748249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 627/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:52.754319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:52.861960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cf51-ef52-4eae-baf6-e5d6ff1c9f2d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:52.904276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:52.904917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:52.905199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:52.905471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:52.906314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] RBAC: Adding query filter params (role.id=8e6112fb4a374ad58c20fbb3e90b17d8, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=41b5461ab9d746b2b45cbc1960dececd) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:52.906598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:52.908035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb6c5-5d6f-42f7-aea8-5f94d567dae4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:52.913914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 619/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:58:52 2022] GET /identity/v3/role_assignments?user.id=41b5461ab9d746b2b45cbc1960dececd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=8e6112fb4a374ad58c20fbb3e90b17d8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:55.144565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e20-c9ef-4f22-86d4-440cef48758b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.145106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e20-c9ef-4f22-86d4-440cef48758b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.145106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3e20-c9ef-4f22-86d4-440cef48758b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.146481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 628/1247] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:55.152513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.152939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.153081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.158708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:55.172548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:55.173761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:55.252665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 620/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:55.279846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3e20-c9ef-4f22-86d4-440cef48758b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:55.301011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:55.397765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3e20-c9ef-4f22-86d4-440cef48758b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:55.436877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:55.437793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.438074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.438349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.451619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:58:55.460920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:55.462007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:55.462894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 629/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:55.469454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:55.582136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af77b3-899a-4072-bf06-4372648bd4c1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:55.614666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:55.621331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:55.622031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.622422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.622731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.623589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:55.623953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:55.625685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:55.641130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 621/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:58:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:55.642485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:55.732577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:55.757998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c071c-aeda-4f1f-b9af-e0809abd66c2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:55.805125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:55.806031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.806309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.806580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.819133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:55.820944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:55.839482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:55.840677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:55.841912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:55.841912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:55.841912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:55.841912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:55.841912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 630/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:55.847000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:55.954540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17888-4a78-4c68-ad12-42ca34b407c6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:55.994863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:55.995758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:55.996120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:55.996595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:55.998434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:55.998847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:56.000769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:56.019145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 622/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:58:56.025252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:56.134111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d24bf6-222e-4e2b-8d87-e82aa2d6c6ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:56.163506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:56.172636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:56.173439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:56.173738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:56.174031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:56.182447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:56.183927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:58:56.199019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:56.200159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:56.201558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 631/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:56.206185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:56.268418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:56.320668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82935c-652a-4675-a319-8bfd3e8881a2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:56.364107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:56.364780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:56.365120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:56.365329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:56.366206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:56.366519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:56.368027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:56.388197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 623/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:58:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:58:56.394245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:56.518259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e53032-96ad-436f-8c13-21559b3aeb79 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:56.556268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:56.556832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:56.557113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:56.557386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:56.616400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:56.629258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:56.630740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:56.743341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 632/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:58:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:58:56.745367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:56.849713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c72a9-39c5-4898-b1b9-1d1095699762 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:56.864096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:56.886179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:56.886995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:56.887278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:56.887553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:56.897740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:56.899202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:58:56.917735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:56.918863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:56.925353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:58:56.925353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:58:56.925353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:58:56.925353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:56.925353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 624/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:58:56.925641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:57.026334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:57.040404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0b59f-e860-4e4d-bf87-6274df06d5d3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:57.077057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:57.077716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:57.078047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:57.078349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:57.079260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:57.079516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:57.081110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:57.094419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 633/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:58:57.100340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:57.208648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea99f47-58dc-4725-865c-c0854fc54139 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:57.251581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:57.252678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:57.252992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:57.253358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:57.261473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:57.263454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:58:57.282071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:57.283203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:58:57.284109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 625/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:58:57.291516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:57.403548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db67ea-2c7b-4dc5-9f5e-6cdfa943ea3f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:57.439969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:57.440659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:57.440947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:57.441225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:57.442032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:57.442347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:57.443896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a25637-0e7d-419b-b5f5-1c6ea3bece5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:57.458268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 634/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:57.464634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:57.492640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:57.573262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a77c9-92d2-4cde-a30c-1a6e9ccd68d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:57.608682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:57.609340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:57.609623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:57.609897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:57.610730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=41b5461ab9d746b2b45cbc1960dececd) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:58:57.611070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:58:57.612501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25753791-37a1-4721-afd0-d361872cc4b8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:58:57.618311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 626/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:58:57 2022] GET /identity/v3/role_assignments?user.id=41b5461ab9d746b2b45cbc1960dececd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:58:59.612507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c47e-7437-4d4a-8e2e-9d4a26939823 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:59.613039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c47e-7437-4d4a-8e2e-9d4a26939823 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:59.613039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c47e-7437-4d4a-8e2e-9d4a26939823 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:59.614404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 635/1261] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:58:59.619485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:59.619849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:59.620039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:59.636837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:58:59.638647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:58:59.701978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 627/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:58:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:58:59.724464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c47e-7437-4d4a-8e2e-9d4a26939823 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:59.734857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:59.822743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c47e-7437-4d4a-8e2e-9d4a26939823 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:58:59.856066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:58:59.857450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:58:59.857732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:58:59.858010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:58:59.867680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:58:59.875104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:58:59.876272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:58:59.877318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 636/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:58:59.882076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:58:59.899762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:58:59.988380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344eb23-4fe5-4776-bd69-876700295782 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.023507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.024171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.024457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.024736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.025485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:00.025793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.027493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.036148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 628/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:00.042289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.105897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:00.138837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d7a51-8589-4ebf-a652-aaa2d7857e00 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.170945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.171716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.171996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.172267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.182465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.183937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:59:00.203027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:00.204071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:00.205345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:59:00.205345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:59:00.205345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:00.205345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:00.205345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 637/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:00.210036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.288440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:00.306431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39837078-e286-4bc8-a17c-f23c50874209 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.339271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.339913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.340192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.340463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.341172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:00.341473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.342842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.354177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 629/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:00.359784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.459061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57ebc8-c602-4f41-815a-03598acf3ea7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.477519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:00.491144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.491927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.492208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.492481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.500066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.501483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:59:00.515609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:00.516726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:00.517720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 638/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:59:00.522498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.616673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87e002-cb3e-4ac4-b6aa-5756fa5aec65 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.649390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.650031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.650311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.650584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.651437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:00.651743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.653150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.661316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:00.670745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 630/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:00.676250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.772111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b2f7f-cf36-4fe1-8f4e-406aa28b0ab7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.803893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.804647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.804931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.805208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.816072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.817486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:59:00.833797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:00.834826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:00.835814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:59:00.835814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:59:00.835814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:00.835814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:00.835814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 639/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:00.840471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:00.933663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc646c-b77c-4ceb-a428-8715247e3bd2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:00.968026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:00.968666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:00.968948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:00.969279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:00.970358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:00.970733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:00.972507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:00.992458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 631/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:00.993622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:01.024810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:01.090251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfa226-04d3-4fe4-8e46-468410a319df None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:01.122480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:01.123278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:01.123561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:01.123839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:01.130456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:01.131946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:59:01.146129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:01.147258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:01.148557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 640/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:59:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:01.152994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:01.211149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:01.252222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397ce82-6f12-4908-8ff8-ac8be8eb9347 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:01.284247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:01.284841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:01.285124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:01.285398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:01.286173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:01.286481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:01.287924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ea4cd-a695-4394-9cdb-6bfe5e97ec88 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:01.300659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 632/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:01.306507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:01.393223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:01.401946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dfe90-f176-45bb-b92f-50e557b011ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:01.435164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:01.436064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:01.436346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:01.436621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] PATH_INFO: `/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/41b5461ab9d746b2b45cbc1960dececd/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:01.438580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] RBAC: Authorizing `identity:create_grant(project_id=269f65c9d92249e5a62bbf25c237b2a8, user_id=41b5461ab9d746b2b45cbc1960dececd, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:01.440032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e96720-2ed2-42ff-b67c-38173c40ab42 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:01.453401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 641/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Fri Jul 1 06:59:01 2022] PUT /identity/v3/projects/269f65c9d92249e5a62bbf25c237b2a8/users/41b5461ab9d746b2b45cbc1960dececd/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 06:59:03.446161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6294a-c6f1-4987-9e4a-f6d8f9d91a67 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:03.446675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6294a-c6f1-4987-9e4a-f6d8f9d91a67 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:03.446763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6294a-c6f1-4987-9e4a-f6d8f9d91a67 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:03.448099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 633/1274] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:59:03.453233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:03.453654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:03.453793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:03.470909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:59:03.472033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:59:03.536769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 642/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:59:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:59:03.560961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6294a-c6f1-4987-9e4a-f6d8f9d91a67 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:03.657816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6294a-c6f1-4987-9e4a-f6d8f9d91a67 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:03.674694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:03.691717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:03.692490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:03.692774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:03.693053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:03.703295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 01 06:59:03.710801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:03.711885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 01 06:59:03.713019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:03.717379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 634/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Fri Jul 1 06:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:03.718525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:03.795245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:03.813948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f764fbf-1c10-403d-9fcc-de15866eab22 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:03.846946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:03.847601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:03.847885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:03.848163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:03.848911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:03.849220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:03.850567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:03.858351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 643/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Fri Jul 1 06:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:59:03.863778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:03.965155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553748e-6c04-49a8-99b0-0ef4c0adc1dd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:03.998508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:03.999320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:03.999606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:03.999884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.009982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.011441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:59:04.028982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:04.030057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:04.031201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:59:04.031201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:59:04.031201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:04.031201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:04.031201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 635/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:04.035822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:04.130717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a68ff-5ed2-4e7b-9b45-f684b688bd46 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:04.162489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:04.163154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:04.163441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:04.163718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.164432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:04.164737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.166082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.177413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 644/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:04.183255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:04.214885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:04.283379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419ce9-b587-44ae-b00e-aa16b3f8d4d5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:04.315871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:04.316655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:04.316937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:04.317210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.324389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.326415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 01 06:59:04.340207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:04.341307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:04.342471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 636/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 01 06:59:04.347153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:04.350822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:04.443112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ea48-9b25-4c2b-9b02-55b83758253a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:04.474508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:04.475217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:04.475497 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:04.475769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.476588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:04.476893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.478307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.496004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 645/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:04.502266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:04.589356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:04.598954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae84d27-b462-4a13-b208-33f429627b8d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:04.631676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:04.632305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:04.632593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:04.632873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.697263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.698646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.782635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:04.792885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 637/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 01 06:59:04.799029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:04.858778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:04.895049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf6e4-962e-46be-8396-52ab3b954363 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:04.927872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:04.928646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:04.928924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:04.929192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:04.938960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:04.940386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 01 06:59:04.958137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:04.959395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:04.960908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 01 06:59:04.960908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 01 06:59:04.960908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 01 06:59:04.960908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:04.960908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 646/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:04.965288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:05.063016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903902b-be1b-4350-b440-1123b5843dc1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:05.095035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:05.095683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:05.095968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:05.096239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:05.096949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:05.097254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:05.098593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:05.109751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 638/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Fri Jul 1 06:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:59:05.115284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:05.208845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f37da4-302e-48e8-a219-65ca47fcbe74 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:05.222756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:05.242152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:05.242962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:05.243243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:05.243517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:05.249869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:05.251339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 01 06:59:05.264917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:05.265965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 01 06:59:05.267138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 647/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Fri Jul 1 06:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:05.271837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:05.319968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:05.368169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb4b47-036d-454c-a5f5-28f724013f10 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:05.400346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:05.400994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:05.401282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:05.401564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:05.402344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:05.402657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:05.404741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31188cb6-1d69-4ae4-8b54-d373a7dccab7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:05.416263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 639/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Fri Jul 1 06:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 06:59:05.422074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:05.519263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96199791-7bff-4a31-aece-aba9d822f081 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:05.551494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:05.552138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:05.552425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:05.552710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:05.553528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] RBAC: Adding query filter params (role.id=c003a22b06be43a186fa3414e2b93df2, scope.project.id=269f65c9d92249e5a62bbf25c237b2a8, user.id=41b5461ab9d746b2b45cbc1960dececd) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:05.553842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:05.555297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362fe-cf8d-4331-9d3c-18aa0e0e0bf8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:05.561667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 648/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Fri Jul 1 06:59:05 2022] GET /identity/v3/role_assignments?user.id=41b5461ab9d746b2b45cbc1960dececd&scope.project.id=269f65c9d92249e5a62bbf25c237b2a8&role.id=c003a22b06be43a186fa3414e2b93df2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:07.588079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0dd82-f48b-4100-8bda-1193d82c2ac5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:07.588635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0dd82-f48b-4100-8bda-1193d82c2ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:07.588635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0dd82-f48b-4100-8bda-1193d82c2ac5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:07.589902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 640/1288] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:59:07.595057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:07.595480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:07.595623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:07.612500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:59:07.613697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:59:07.642333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:07.678343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 649/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:59:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:59:07.701264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0dd82-f48b-4100-8bda-1193d82c2ac5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:07.800032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0dd82-f48b-4100-8bda-1193d82c2ac5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:07.810401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:07.832972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:07.833754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:07.834037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:07.834312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:07.834997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:07.836350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:07.848355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:07.849597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:07.850905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 641/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:59:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:07.855221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:07.950164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7dba-2bd5-410b-ac33-b4301ac44296 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:07.989135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:07.989811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:07.990088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:07.990364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:07.991144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:07.991430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:07.992994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75209710-efa7-4e47-8dbe-94478180626e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:07.998103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 650/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Jul 1 06:59:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:08.003931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:08.101196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a857b-1ff4-43f9-add6-ddad565476e1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:08.133315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:08.133950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:08.134229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:08.134501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:08.135223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:08.135524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:08.136868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fd499-ad23-4ecb-8e97-79a6778e567b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:08.141137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 642/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Jul 1 06:59:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:10.119459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3d1-fa0a-454a-afb5-49f22eeb1251 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:10.120024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3d1-fa0a-454a-afb5-49f22eeb1251 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:10.120024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3d1-fa0a-454a-afb5-49f22eeb1251 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:10.121386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 651/1293] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:59:10.127138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6c8c-4a6b-48cb-8292-80e75e1c81c2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:10.127528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6c8c-4a6b-48cb-8292-80e75e1c81c2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:10.127693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6c8c-4a6b-48cb-8292-80e75e1c81c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:10.144500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aace6c8c-4a6b-48cb-8292-80e75e1c81c2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:59:10.145669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aace6c8c-4a6b-48cb-8292-80e75e1c81c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:59:10.173514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:10.211622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 643/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:59:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:59:10.234346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbad3d1-fa0a-454a-afb5-49f22eeb1251 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:10.264973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:10.332853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbad3d1-fa0a-454a-afb5-49f22eeb1251 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:10.364415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:10.365054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:10.365333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:10.365604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:10.366340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:10.367716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ed247-ebf4-49ed-8e89-ae57f8862b0e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:10.380251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 652/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Fri Jul 1 06:59:10 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:59:12.421817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ad367-36ae-4171-9de4-5706aeb190af None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:12.422351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ad367-36ae-4171-9de4-5706aeb190af None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:12.422351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ad367-36ae-4171-9de4-5706aeb190af None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:12.423758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 644/1296] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:59:12.428893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:12.429326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:12.429426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:12.445753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:59:12.446950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:59:12.515973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 653/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:59:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 01 06:59:12.538954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ad367-36ae-4171-9de4-5706aeb190af None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:12.635439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ad367-36ae-4171-9de4-5706aeb190af None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:12.667613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:12.668435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:12.668716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:12.668990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:12.669655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:12.671560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:12.683082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:12.684256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:12.685366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 645/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:59:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:12.690379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:12.708621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:12.786891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16ff3-d2d1-4ec4-8aad-48e404c84cbf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:12.818413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:12.819079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:12.819360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:12.819634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:12.820324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:12.820624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:12.821963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:12.827756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 654/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Jul 1 06:59:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:12.833609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:12.864638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:12.930724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86975b6-1a2f-44da-9814-9490abb5fe01 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:12.965342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:12.965993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:12.966283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:12.966562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:12.967383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] RBAC: Adding query filter params (interface=public, service_id=d6f6d33cf9c5486e884841262b828fa8, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:12.967708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:12.969626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab8b72-94dc-4289-8e70-d3a6a26685a9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:12.974729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 646/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Fri Jul 1 06:59:12 2022] GET /identity/v3/endpoints?service_id=d6f6d33cf9c5486e884841262b828fa8&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:12.980947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:13.066718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:13.084619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efce51e-ede2-402d-8096-f50579d2afb0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:13.116145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:13.116789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:13.117068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:13.117342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:13.118009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:13.119391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff8c04-6322-45ea-b119-057e10bf678f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:13.126432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 655/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Fri Jul 1 06:59:12 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 06:59:15.118533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f31f-a66f-42af-b22c-a32801bb6860 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:15.119118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f31f-a66f-42af-b22c-a32801bb6860 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:15.119118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70f31f-a66f-42af-b22c-a32801bb6860 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:15.120486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 647/1302] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 06:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 06:59:15.126098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:15.126470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:15.126649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:15.142692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 06:59:15.143884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 06:59:15.206794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 656/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 06:59:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 01 06:59:15.229590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70f31f-a66f-42af-b22c-a32801bb6860 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:15.322883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:15.329858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70f31f-a66f-42af-b22c-a32801bb6860 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:15.363980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:15.364760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:15.365041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:15.365317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:15.365966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:15.367363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 01 06:59:15.378930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 01 06:59:15.380113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 01 06:59:15.381140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 648/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Fri Jul 1 06:59:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 01 06:59:15.386166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:15.458208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:15.481275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44701f2c-c507-4b16-93c2-aaa8b202076e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:15.513729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:15.514367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:15.514648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:15.514949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:15.515647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 06:59:15.515953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:15.517336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9bf43-20b1-4e2a-949c-b3f9c14ec710 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:15.522294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 657/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Fri Jul 1 06:59:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 06:59:15.527819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 06:59:15.625209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5382f-0558-4039-a79c-ea4a6f9b209c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 06:59:15.657868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 06:59:15.658452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 06:59:15.658737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 06:59:15.659047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 06:59:15.659796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 06:59:15.661147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c77c5b-95ae-4396-924f-2257907e5237 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 06:59:15.679072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 649/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Fri Jul 1 06:59:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 06:59:25.519416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=30 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 06:59:25.674858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:17.892857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d399-47fb-4bce-9982-1e6e7f9d1842 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:17.893977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d399-47fb-4bce-9982-1e6e7f9d1842 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:17.894488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8d399-47fb-4bce-9982-1e6e7f9d1842 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:17.901649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d88f9-a133-41f0-93a0-12a37ae655b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:17.903094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d88f9-a133-41f0-93a0-12a37ae655b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:17.903599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d88f9-a133-41f0-93a0-12a37ae655b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:17.905728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 658/1307] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:18.127779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-715d88f9-a133-41f0-93a0-12a37ae655b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:18.128940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-715d88f9-a133-41f0-93a0-12a37ae655b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:18.301162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 650/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:18.352314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:18.352952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:18.353481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:18.355600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 659/1309] 10.222.0.48 () {54 vars in 964 bytes} [Fri Jul 1 07:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:18.361307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cc74-b4f0-406f-b0ac-b0c681b8a51e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:18.361601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cc74-b4f0-406f-b0ac-b0c681b8a51e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:18.361882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cc74-b4f0-406f-b0ac-b0c681b8a51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:18.469873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e24cc74-b4f0-406f-b0ac-b0c681b8a51e None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:18.471157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e24cc74-b4f0-406f-b0ac-b0c681b8a51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:18.494284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:18.559029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 651/1310] 10.222.0.48 () {58 vars in 1061 bytes} [Fri Jul 1 07:00:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:18.568777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:18.570935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:18.696430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3f78f-f480-41cd-b1cf-e9da8fb22ec1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:18.779457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:18.780625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:18.781132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:18.781623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:18.783510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:18.842445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:18.902888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:18.905096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4f01a-bd68-4298-a862-74ff5ba080e7 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:18.936895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 660/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:21.305159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:21.305680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:21.305771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:21.307304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 652/1312] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:21.312669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587bf9f-418d-433b-b8f7-4f068171f8fe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:21.313079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587bf9f-418d-433b-b8f7-4f068171f8fe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:21.313232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587bf9f-418d-433b-b8f7-4f068171f8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:21.381494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7587bf9f-418d-433b-b8f7-4f068171f8fe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:21.382718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7587bf9f-418d-433b-b8f7-4f068171f8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:21.478606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 661/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:21.516799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:21.518393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:21.628272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2958e6a-18bc-4300-ac24-c0e0d04f025d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:21.705775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:21.708887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:21.709558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:21.709860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:21.710301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:21.712309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:21.820567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:21.822011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0673f0-31f5-49ec-aa4d-c98ca7b4591d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:21.841113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 653/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:24.171538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05279d9-956d-4f92-bf8f-ad54ae6a2bb2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:24.172111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05279d9-956d-4f92-bf8f-ad54ae6a2bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:24.172111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05279d9-956d-4f92-bf8f-ad54ae6a2bb2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:24.173449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 662/1315] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:24.178605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60acab5-780c-4c9b-8dde-b937d09555f8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:24.179064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60acab5-780c-4c9b-8dde-b937d09555f8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:24.179211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60acab5-780c-4c9b-8dde-b937d09555f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:24.242951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d60acab5-780c-4c9b-8dde-b937d09555f8 None None] MFA Rules not processed for user `9b03423a023143208d210010093cfe64`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:24.244120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d60acab5-780c-4c9b-8dde-b937d09555f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:24.295499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:24.340633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 654/1316] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:24.394873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:24.395155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:24.395433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:24.396804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 663/1317] 10.222.0.48 () {54 vars in 964 bytes} [Fri Jul 1 07:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:24.402387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049f5d-cac6-4a51-8923-563a3e42e73c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:24.402686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049f5d-cac6-4a51-8923-563a3e42e73c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:24.402994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049f5d-cac6-4a51-8923-563a3e42e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:24.475166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-08049f5d-cac6-4a51-8923-563a3e42e73c None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:24.476385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08049f5d-cac6-4a51-8923-563a3e42e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:24.549745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 655/1318] 10.222.0.48 () {58 vars in 1061 bytes} [Fri Jul 1 07:00:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:24.558943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:24.560755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:24.570157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:24.661910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d4960-ec32-424e-b76b-51cd37b1a54f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:24.734380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:24.735043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:24.735322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:24.735591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:24.737208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:24.796632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:24.841823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:24.843213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054af7f9-3765-4e4a-b281-8ba60b0e9641 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:24.861329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 664/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:27.762897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:27.763436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:27.763436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:27.764832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 656/1320] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:27.770101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194107-5f9e-4b07-b1a1-83af15578c27 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:27.770867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194107-5f9e-4b07-b1a1-83af15578c27 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:27.771148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194107-5f9e-4b07-b1a1-83af15578c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:27.833236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fe194107-5f9e-4b07-b1a1-83af15578c27 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:27.834423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe194107-5f9e-4b07-b1a1-83af15578c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:27.926955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 665/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:27.966241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:27.968394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:28.017789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:28.070598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb997ad3-3524-40a5-aded-48965d895600 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:28.140161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:28.140796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:28.141077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:28.141351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:28.142987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:28.236330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:28.249568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:28.251030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af0428-2936-4a37-9066-4ccfb9911519 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:28.268432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 657/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:30.969752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:30.970316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:30.970316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:30.971775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 666/1323] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:30.977009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d375-964f-4f63-8b6e-4e0abd243bd2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:30.977442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d375-964f-4f63-8b6e-4e0abd243bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:30.977576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646d375-964f-4f63-8b6e-4e0abd243bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:31.043629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8646d375-964f-4f63-8b6e-4e0abd243bd2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:31.044844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8646d375-964f-4f63-8b6e-4e0abd243bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:31.145340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 658/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:31.186002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:31.187614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:31.208186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:31.298003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6ff5-be2d-440d-98e6-84db624093f0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:31.369828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:31.370478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:31.370802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:31.371080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:31.372701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:31.442600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:31.482868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:31.484250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863443f6-e288-48d4-84f8-d1f65ee62440 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:31.502368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 667/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:33.565059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:33.565578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:33.565578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:33.566890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 659/1326] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:33.571957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77192bd-8669-4065-b2cc-6d9b8e1eb6ca None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:33.572393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77192bd-8669-4065-b2cc-6d9b8e1eb6ca None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:33.572538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77192bd-8669-4065-b2cc-6d9b8e1eb6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:33.635453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d77192bd-8669-4065-b2cc-6d9b8e1eb6ca None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:33.636649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d77192bd-8669-4065-b2cc-6d9b8e1eb6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:33.731545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 668/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:33.766756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:33.768324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:33.827759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:33.871099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff76bcc-1dcc-44f2-9fc5-175464311464 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:33.943887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:33.944540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:33.944828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:33.945115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:33.947066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:34.062852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:34.065611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:34.067295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45977e05-f0c6-4e5b-af07-5888ca046ddf service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:34.086288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 660/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:36.162516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:36.163074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:36.163195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:36.164508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 669/1329] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:36.170496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dfb9a-fabc-43a3-a1a2-effac47c923e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:36.170941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dfb9a-fabc-43a3-a1a2-effac47c923e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:36.171083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dfb9a-fabc-43a3-a1a2-effac47c923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:36.240511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-421dfb9a-fabc-43a3-a1a2-effac47c923e None None] MFA Rules not processed for user `9b03423a023143208d210010093cfe64`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:36.241676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-421dfb9a-fabc-43a3-a1a2-effac47c923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:36.341539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 661/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:36 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:36.379165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:36.380757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:36.411190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:36.484789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a8e6e-48b2-4512-aa6d-430432d33f7c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:36.561823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:36.562752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:36.563096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:36.563377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:36.565932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:36.643631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:36.676943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:36.678498 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b24def-3e83-4384-bb22-0c8519f20804 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:36.698004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 670/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:36 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:39.029774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:39.030379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:39.030379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:39.031730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 662/1332] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:39.037057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324167be-6507-4d6a-9dcd-a92b1d03bf4d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:39.037539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324167be-6507-4d6a-9dcd-a92b1d03bf4d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:39.037654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324167be-6507-4d6a-9dcd-a92b1d03bf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:39.105230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-324167be-6507-4d6a-9dcd-a92b1d03bf4d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:39.106440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-324167be-6507-4d6a-9dcd-a92b1d03bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:39.203270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 671/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:39.243018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:39.244578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:39.326491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:39.367846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57801222-45e3-4cb8-8b5e-d6a69762fae0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:39.439727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:39.440363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:39.440645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:39.440918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:39.442540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:39.546127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:39.547573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6201992e-68b8-4938-a979-2416978a5e40 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:39.557696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:39.566913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 663/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:41.611091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:41.611626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:41.611626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:41.613004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 672/1335] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:41.618158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d61ce7-5b98-4bd8-8deb-aacf88651c40 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:41.618603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d61ce7-5b98-4bd8-8deb-aacf88651c40 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:41.618706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d61ce7-5b98-4bd8-8deb-aacf88651c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:41.681479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-39d61ce7-5b98-4bd8-8deb-aacf88651c40 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:41.683190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-39d61ce7-5b98-4bd8-8deb-aacf88651c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:41.776062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 664/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:41.815697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:41.817196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:41.855840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:41.917716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e10ff-ea43-4a0f-ad0b-3d0c562e036c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:41.990193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:41.990866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:41.991147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:41.991418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:41.993046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:42.073035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:42.097795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:42.099262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d5154-f6c5-48a9-8509-778d3f1bd5d0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:42.116974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 673/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:44.439146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:44.439711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:44.439711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:44.441066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 665/1338] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:44.446302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec09-165f-4198-b4b6-eb1e1151562a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:44.446731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec09-165f-4198-b4b6-eb1e1151562a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:44.446869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec09-165f-4198-b4b6-eb1e1151562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:44.509586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-73ffec09-165f-4198-b4b6-eb1e1151562a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:44.511343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73ffec09-165f-4198-b4b6-eb1e1151562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:44.608138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 674/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:44.643938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:44.645483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:44.719431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:44.747559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bda7b-9665-4720-a6cd-400006d47c15 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:44.817116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:44.817752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:44.818032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:44.818304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:44.819951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:44.935652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:44.937085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa667b-0efc-4c54-a4c4-1025642d7205 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:44.954520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 666/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:48.882299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:48.882862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:48.882960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:48.884279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 675/1341] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:48.890260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fc7e-e892-412e-8c1d-e3bd192ba820 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:48.890694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fc7e-e892-412e-8c1d-e3bd192ba820 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:48.891050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fc7e-e892-412e-8c1d-e3bd192ba820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:48.897362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:48.964843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fef1fc7e-e892-412e-8c1d-e3bd192ba820 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:48.966052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fef1fc7e-e892-412e-8c1d-e3bd192ba820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:49.069032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 667/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:49.109364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:49.111509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:49.160068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:00:49.207120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222fa32-c3a6-49fc-897f-6dc1233e1a4d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:49.210622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:49.211298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:49.211584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:49.211864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:49.221167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:49.222598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349dbaf-3721-4612-8c35-16ed9314afc7 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:49.231862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 676/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:52.092838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:52.093382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:52.093382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:52.094982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 668/1344] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:52.100407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b682d9a-fe15-40d7-861b-fb6f6ba21f3b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:52.100893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b682d9a-fe15-40d7-861b-fb6f6ba21f3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:52.101088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b682d9a-fe15-40d7-861b-fb6f6ba21f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:52.130133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9b682d9a-fe15-40d7-861b-fb6f6ba21f3b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:52.131359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b682d9a-fe15-40d7-861b-fb6f6ba21f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:52.215571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 677/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:52.252358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:52.254080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:52.267868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf587a-0dce-4c5a-a777-6e813af6153e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:52.271379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:52.272012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:52.272298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:52.272578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:52.281419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:52.282879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc321c3-a104-4bef-bc99-c09c994edddc service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:52.292237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 669/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:00:56.325967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb5d6-441a-4cee-8ce7-e918b2fe1e88 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:56.326482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb5d6-441a-4cee-8ce7-e918b2fe1e88 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:56.326604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afb5d6-441a-4cee-8ce7-e918b2fe1e88 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:56.328090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 678/1347] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:00:56.333194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6d3a4-3f99-4d71-a7bd-5785aeca9fbf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:56.333617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6d3a4-3f99-4d71-a7bd-5785aeca9fbf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:56.333814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6d3a4-3f99-4d71-a7bd-5785aeca9fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:56.351561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c2c6d3a4-3f99-4d71-a7bd-5785aeca9fbf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:00:56.352761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2c6d3a4-3f99-4d71-a7bd-5785aeca9fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:00:56.360695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 670/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:00:56.397813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afb5d6-441a-4cee-8ce7-e918b2fe1e88 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:00:56.409151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afb5d6-441a-4cee-8ce7-e918b2fe1e88 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:00:56.412593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:00:56.413229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:00:56.413505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:00:56.413804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:00:56.423677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:00:56.425118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a3e97-739c-40d1-a6f8-26e171f6d186 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:00:56.435793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 679/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:01.402268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79844dfc-86ca-4f5f-a5d1-1026555ade8c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:01.403838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79844dfc-86ca-4f5f-a5d1-1026555ade8c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:01.403838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79844dfc-86ca-4f5f-a5d1-1026555ade8c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:01.405172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 671/1350] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:01.410388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c2ffd-6202-4dcf-a5d3-8b28ce998a61 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:01.411274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c2ffd-6202-4dcf-a5d3-8b28ce998a61 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:01.411751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c2ffd-6202-4dcf-a5d3-8b28ce998a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:01.492671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-206c2ffd-6202-4dcf-a5d3-8b28ce998a61 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:01.493878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-206c2ffd-6202-4dcf-a5d3-8b28ce998a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:01.565361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:01.634670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 680/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:01:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:01:05.745832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c32d9-7032-44c9-abd5-28bde564de4e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:05.746706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c32d9-7032-44c9-abd5-28bde564de4e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:05.747182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c32d9-7032-44c9-abd5-28bde564de4e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:05.748536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 672/1352] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:05.753993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a855-c951-4780-b14f-f50073c67944 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:05.754431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a855-c951-4780-b14f-f50073c67944 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:05.754533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8a855-c951-4780-b14f-f50073c67944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:05.773482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ef8a855-c951-4780-b14f-f50073c67944 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:05.774752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ef8a855-c951-4780-b14f-f50073c67944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:05.783259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 681/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:01:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:06.346906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=72 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:07.376306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87c08-63f2-4098-8ff5-388268bbe23f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:07.376907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87c08-63f2-4098-8ff5-388268bbe23f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:07.376907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87c08-63f2-4098-8ff5-388268bbe23f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:07.378324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 673/1354] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:07.383479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815cfa-6dbf-45b2-a308-18db1cab390f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:07.383905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815cfa-6dbf-45b2-a308-18db1cab390f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:07.384158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12815cfa-6dbf-45b2-a308-18db1cab390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:07.402573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-12815cfa-6dbf-45b2-a308-18db1cab390f None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:07.403780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-12815cfa-6dbf-45b2-a308-18db1cab390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:07.413932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 682/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:01:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:10.905485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6372a1-3d27-43cc-a247-f0269b8dfad2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:10.906022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6372a1-3d27-43cc-a247-f0269b8dfad2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:10.906022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6372a1-3d27-43cc-a247-f0269b8dfad2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:10.907529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 674/1356] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:10.913624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829386e-694d-4437-98e2-0a32a73f611a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:10.914153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829386e-694d-4437-98e2-0a32a73f611a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:10.914620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829386e-694d-4437-98e2-0a32a73f611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:10.932983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b829386e-694d-4437-98e2-0a32a73f611a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:10.934157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b829386e-694d-4437-98e2-0a32a73f611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:10.942913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 683/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:11.000380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215f9a-b6da-448c-b7c5-fbd4b34c5578 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:11.001019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215f9a-b6da-448c-b7c5-fbd4b34c5578 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:11.001472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c215f9a-b6da-448c-b7c5-fbd4b34c5578 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:11.003152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 675/1358] 10.222.0.48 () {54 vars in 961 bytes} [Fri Jul 1 07:01:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:11.013513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df84e1b-4912-4011-a3ae-371c15266559 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:11.014112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df84e1b-4912-4011-a3ae-371c15266559 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:11.014682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df84e1b-4912-4011-a3ae-371c15266559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:11.069091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7df84e1b-4912-4011-a3ae-371c15266559 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:11.070330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7df84e1b-4912-4011-a3ae-371c15266559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:11.160427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 684/1359] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Jul 1 07:01:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:11.168914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215f9a-b6da-448c-b7c5-fbd4b34c5578 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:11.185261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c215f9a-b6da-448c-b7c5-fbd4b34c5578 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:11.196512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:11.197171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:11.198084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:11.198361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:11.207739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:11.209561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc644645-2921-4660-ac11-6b08ec4912c3 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:11.220824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 676/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:13.438855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71de7-f219-4c87-916b-e3ed4c36fdde None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:13.439400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71de7-f219-4c87-916b-e3ed4c36fdde None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:13.439400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a71de7-f219-4c87-916b-e3ed4c36fdde None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:13.440774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 685/1361] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:13.445822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713baee-6bcb-41d6-b39a-0cf727c1b89c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:13.446228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713baee-6bcb-41d6-b39a-0cf727c1b89c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:13.446358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713baee-6bcb-41d6-b39a-0cf727c1b89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:13.465300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e713baee-6bcb-41d6-b39a-0cf727c1b89c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:13.466759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e713baee-6bcb-41d6-b39a-0cf727c1b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:13.474585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 677/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:13.518356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a71de7-f219-4c87-916b-e3ed4c36fdde None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:13.533613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a71de7-f219-4c87-916b-e3ed4c36fdde None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:13.539065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:13.539695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:13.539981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:13.540261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:13.548668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:13.550273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83bb19-2e05-4203-8640-5bc58be4b9d3 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:13.560248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 686/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:15.894964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca066-cea9-46d0-89c9-661d6a62b186 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:15.895504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca066-cea9-46d0-89c9-661d6a62b186 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:15.895605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ca066-cea9-46d0-89c9-661d6a62b186 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:15.897066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 678/1364] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:15.901964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca676f-a830-4bb0-8995-ad113a7213ae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:15.902382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca676f-a830-4bb0-8995-ad113a7213ae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:15.902519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca676f-a830-4bb0-8995-ad113a7213ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:15.922108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9cca676f-a830-4bb0-8995-ad113a7213ae None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:15.923617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9cca676f-a830-4bb0-8995-ad113a7213ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:15.932553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 687/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:15.981003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1849-7d94-40f3-a6f2-da03f71cc56c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:15.981295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1849-7d94-40f3-a6f2-da03f71cc56c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:15.981564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a1849-7d94-40f3-a6f2-da03f71cc56c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:15.983213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 679/1366] 10.222.0.48 () {54 vars in 961 bytes} [Fri Jul 1 07:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:15.993575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93099-6bd4-4fc8-9929-781c59a5b463 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:15.993930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93099-6bd4-4fc8-9929-781c59a5b463 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:15.994905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d93099-6bd4-4fc8-9929-781c59a5b463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:16.015223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-04d93099-6bd4-4fc8-9929-781c59a5b463 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:16.016489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04d93099-6bd4-4fc8-9929-781c59a5b463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:16.026382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 688/1367] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Jul 1 07:01:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:16.034432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a1849-7d94-40f3-a6f2-da03f71cc56c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:16.049684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a1849-7d94-40f3-a6f2-da03f71cc56c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:16.055141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:16.055865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:16.056217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:16.056559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:16.064887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:16.066526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1172c0-1cbc-49a6-80ad-cd550475ef06 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:16.075949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 680/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:18.182921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35564106-6d32-4c24-9f46-d6c42c2f124f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:18.183510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35564106-6d32-4c24-9f46-d6c42c2f124f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:18.183510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35564106-6d32-4c24-9f46-d6c42c2f124f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:18.185113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 689/1369] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:18.190469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa832-13b1-492f-a135-5231608060cd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:18.190922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa832-13b1-492f-a135-5231608060cd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:18.191097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffa832-13b1-492f-a135-5231608060cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:18.210895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-57ffa832-13b1-492f-a135-5231608060cd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:18.212874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57ffa832-13b1-492f-a135-5231608060cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:18.222286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 681/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:18.266127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35564106-6d32-4c24-9f46-d6c42c2f124f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:18.282434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35564106-6d32-4c24-9f46-d6c42c2f124f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:18.288479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:18.289334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:18.289718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:18.290096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:18.300528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:18.302310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0172647-9008-426b-b356-c6fbb9cea49e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:18.314293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 690/1371] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:19.050589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2ee7-91ad-405e-b384-f031470ed052 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:19.050948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2ee7-91ad-405e-b384-f031470ed052 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:19.051234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2ee7-91ad-405e-b384-f031470ed052 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:19.052614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 682/1372] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:19.058426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25606-eb57-4cf4-a060-69dd4c2c92b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:19.058718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25606-eb57-4cf4-a060-69dd4c2c92b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:19.059029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25606-eb57-4cf4-a060-69dd4c2c92b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:19.077795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-68f25606-eb57-4cf4-a060-69dd4c2c92b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:19.079025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68f25606-eb57-4cf4-a060-69dd4c2c92b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:19.087042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 691/1373] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:19.424408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462442-8178-4eda-9a43-a7993e309368 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:19.424408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462442-8178-4eda-9a43-a7993e309368 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:19.424408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462442-8178-4eda-9a43-a7993e309368 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:19.427412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 683/1374] 10.222.0.48 () {54 vars in 963 bytes} [Fri Jul 1 07:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:19.430742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90048fa3-d694-4934-bf07-9ca62e2b4591 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:19.431213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90048fa3-d694-4934-bf07-9ca62e2b4591 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:19.432200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90048fa3-d694-4934-bf07-9ca62e2b4591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:19.440593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:19.486108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90048fa3-d694-4934-bf07-9ca62e2b4591 None None] MFA Rules not processed for user `52d7bcc4b67c495f821519996cc2ab26`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:19.487352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90048fa3-d694-4934-bf07-9ca62e2b4591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:19.554909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 692/1375] 10.222.0.48 () {58 vars in 1060 bytes} [Fri Jul 1 07:01:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:01:19.557673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e462442-8178-4eda-9a43-a7993e309368 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:19.568828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e462442-8178-4eda-9a43-a7993e309368 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:19.572429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:19.573067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:19.573352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:19.573630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:19.583122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:19.584540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba81f1d-bd08-4c10-bf25-022c5becc1d0 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:19.594226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 684/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:20.375051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace991be-5d5c-457f-b278-33f1c9e71018 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:20.375348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace991be-5d5c-457f-b278-33f1c9e71018 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:20.375630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace991be-5d5c-457f-b278-33f1c9e71018 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:20.377012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 693/1377] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 07:01:20.383118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598f31d-0462-49db-9b40-fca7b708e513 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:20.383611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598f31d-0462-49db-9b40-fca7b708e513 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:20.383962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598f31d-0462-49db-9b40-fca7b708e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:20.405647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0598f31d-0462-49db-9b40-fca7b708e513 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:20.406977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0598f31d-0462-49db-9b40-fca7b708e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:20.415064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 685/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:20.458695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace991be-5d5c-457f-b278-33f1c9e71018 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:20.473538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace991be-5d5c-457f-b278-33f1c9e71018 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:20.479122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:20.479903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:20.480254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:20.480598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:20.488799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:20.490376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65754fac-102d-499d-8d04-b358cc6848af service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:20.499533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 694/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:22.597150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677eb8d-e389-4165-b1b7-7d41aba321a9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:22.597642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677eb8d-e389-4165-b1b7-7d41aba321a9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:22.597742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677eb8d-e389-4165-b1b7-7d41aba321a9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:22.599269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 686/1380] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:22.605135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791118f-b9f7-4aea-8afd-36e5cbbbb66c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:22.605548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791118f-b9f7-4aea-8afd-36e5cbbbb66c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:22.605688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791118f-b9f7-4aea-8afd-36e5cbbbb66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:22.625928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3791118f-b9f7-4aea-8afd-36e5cbbbb66c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:22.627193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3791118f-b9f7-4aea-8afd-36e5cbbbb66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:22.635041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 695/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:22.680970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677eb8d-e389-4165-b1b7-7d41aba321a9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:22.702853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2677eb8d-e389-4165-b1b7-7d41aba321a9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:22.708707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:22.709425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:22.709780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:22.710123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:22.719070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:22.720598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d15c3-8a28-434d-97d1-e9c99246c1ba service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:22.730655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 687/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:24.794393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b11137-ee46-4cf7-9bb4-8a67d03e2c9e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:24.794978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b11137-ee46-4cf7-9bb4-8a67d03e2c9e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:24.794978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b11137-ee46-4cf7-9bb4-8a67d03e2c9e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:24.796335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 696/1383] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:24.801778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a34ac-1840-4f19-9f1d-ae69bbaf781b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:24.802194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a34ac-1840-4f19-9f1d-ae69bbaf781b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:24.802340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a34ac-1840-4f19-9f1d-ae69bbaf781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:24.821885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c34a34ac-1840-4f19-9f1d-ae69bbaf781b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:24.823320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c34a34ac-1840-4f19-9f1d-ae69bbaf781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:24.831307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 688/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:24.873324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b11137-ee46-4cf7-9bb4-8a67d03e2c9e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:24.887728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b11137-ee46-4cf7-9bb4-8a67d03e2c9e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:24.893183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:24.893949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:24.894370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:24.894746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:24.903275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:24.904960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25275577-dceb-4d91-8827-614702f6d7a5 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:24.914722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 697/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:26.969946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c4bc9-1c52-471d-830f-e3a10d5ca20e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:26.970441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c4bc9-1c52-471d-830f-e3a10d5ca20e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:26.970561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c4bc9-1c52-471d-830f-e3a10d5ca20e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:26.972254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 689/1386] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:26.977203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad884e4-7420-4bf1-a995-9000f553ffff None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:26.977618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad884e4-7420-4bf1-a995-9000f553ffff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:26.977771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad884e4-7420-4bf1-a995-9000f553ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:26.998826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bad884e4-7420-4bf1-a995-9000f553ffff None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:27.000186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bad884e4-7420-4bf1-a995-9000f553ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:27.011016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 698/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:27.052995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c4bc9-1c52-471d-830f-e3a10d5ca20e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:27.074293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c4bc9-1c52-471d-830f-e3a10d5ca20e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:27.086540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:27.088822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:27.089483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:27.090490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:27.110154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:27.113890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189c25d-202a-4e45-84c8-1473c453665d service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:27.133808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 690/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:29.260625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f6af-3286-4f91-8b78-a9d6e84c6c31 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:29.261740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f6af-3286-4f91-8b78-a9d6e84c6c31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:29.261740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4f6af-3286-4f91-8b78-a9d6e84c6c31 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:29.263649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 699/1389] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:29.270176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfef4-b291-4c91-8d0d-1529bad784e7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:29.270616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfef4-b291-4c91-8d0d-1529bad784e7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:29.270941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cfef4-b291-4c91-8d0d-1529bad784e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:29.294055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-610cfef4-b291-4c91-8d0d-1529bad784e7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:29.295688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-610cfef4-b291-4c91-8d0d-1529bad784e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:29.305767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 691/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:29.351618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4f6af-3286-4f91-8b78-a9d6e84c6c31 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:29.371486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4f6af-3286-4f91-8b78-a9d6e84c6c31 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:29.378367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:29.379202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:29.379832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:29.380433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:29.391358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:29.393432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d27cf-4178-4fc1-bf24-eb5d9987efec service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:29.406076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 700/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:31.544500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146987-35f8-4416-8733-f05b22132f80 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:31.545017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146987-35f8-4416-8733-f05b22132f80 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:31.545017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146987-35f8-4416-8733-f05b22132f80 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:31.546341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 692/1392] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:31.551980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dcd81-8a5e-4755-b95c-a25b04ef30e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:31.552423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dcd81-8a5e-4755-b95c-a25b04ef30e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:31.552715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796dcd81-8a5e-4755-b95c-a25b04ef30e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:31.575758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-796dcd81-8a5e-4755-b95c-a25b04ef30e1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:31.577363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-796dcd81-8a5e-4755-b95c-a25b04ef30e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:31.587623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 701/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:31.633712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146987-35f8-4416-8733-f05b22132f80 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:31.648325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146987-35f8-4416-8733-f05b22132f80 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:31.654441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:31.655107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:31.655395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:31.655717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:31.664281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:31.665899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a974bfb-4bd8-4caf-89f8-abb5c363412a service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:31.675848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 693/1394] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:01:33.853854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5ae8-be9f-4b51-9fb3-eb3dc1770319 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:33.854422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5ae8-be9f-4b51-9fb3-eb3dc1770319 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:33.854422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd5ae8-be9f-4b51-9fb3-eb3dc1770319 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:33.855916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 702/1395] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:33.861556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966081d9-de5f-49fa-94c0-a2a43d667c5c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:33.862034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966081d9-de5f-49fa-94c0-a2a43d667c5c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:33.862146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966081d9-de5f-49fa-94c0-a2a43d667c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:33.879526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-966081d9-de5f-49fa-94c0-a2a43d667c5c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:33.880718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-966081d9-de5f-49fa-94c0-a2a43d667c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:33.889194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 694/1396] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:33.931906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd5ae8-be9f-4b51-9fb3-eb3dc1770319 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:33.946476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd5ae8-be9f-4b51-9fb3-eb3dc1770319 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:33.952548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:33.953175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:33.953456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:33.953750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:33.961782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:33.964421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6961464-6380-475b-bff3-9a602c5dcbe2 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:33.974238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 703/1397] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:34.786433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec21db-c760-4245-96fc-cc52406bbd5b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:34.786721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec21db-c760-4245-96fc-cc52406bbd5b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:34.787027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec21db-c760-4245-96fc-cc52406bbd5b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:34.788390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 695/1398] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:34.794338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372a393-0683-4f0d-ae6f-6edce29b444c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:34.794626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372a393-0683-4f0d-ae6f-6edce29b444c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:34.794941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372a393-0683-4f0d-ae6f-6edce29b444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:34.813622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a372a393-0683-4f0d-ae6f-6edce29b444c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:34.814936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a372a393-0683-4f0d-ae6f-6edce29b444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:34.826935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 704/1399] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:34.880533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4ff7-03e9-428a-ab24-0d8de770a039 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:34.881085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4ff7-03e9-428a-ab24-0d8de770a039 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:34.881266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4ff7-03e9-428a-ab24-0d8de770a039 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:34.888122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 696/1400] 10.222.0.48 () {54 vars in 962 bytes} [Fri Jul 1 07:01:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:34.895746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810c984-1e8a-47d4-ab8b-4b426dd441a1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:34.896456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810c984-1e8a-47d4-ab8b-4b426dd441a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:34.896613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810c984-1e8a-47d4-ab8b-4b426dd441a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:34.947842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6810c984-1e8a-47d4-ab8b-4b426dd441a1 None None] MFA Rules not processed for user `f1d06e6ad60240329d83bc4767439ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:34.949022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6810c984-1e8a-47d4-ab8b-4b426dd441a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:34.981231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:35.009724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 705/1401] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Jul 1 07:01:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:35.017571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b4ff7-03e9-428a-ab24-0d8de770a039 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:35.028596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b4ff7-03e9-428a-ab24-0d8de770a039 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:35.032134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:35.032768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:35.033045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:35.033320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:35.042794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:35.044160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d3dce-98c2-47f9-9e3b-6acf36c414f3 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:35.053562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 697/1402] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:35.758800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8bc1-6fc9-411b-ad2b-fb71c6f99cf5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:35.759094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8bc1-6fc9-411b-ad2b-fb71c6f99cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:35.759982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac8bc1-6fc9-411b-ad2b-fb71c6f99cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:35.805456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7bac8bc1-6fc9-411b-ad2b-fb71c6f99cf5 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:35.807217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7bac8bc1-6fc9-411b-ad2b-fb71c6f99cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:35.872305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 706/1403] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:01:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:35.918032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa0fe-1a3c-4586-9dc7-283d8044d783 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:35.918487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa0fe-1a3c-4586-9dc7-283d8044d783 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:35.918735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780aa0fe-1a3c-4586-9dc7-283d8044d783 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:35.920296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 698/1404] 10.222.0.48 () {54 vars in 942 bytes} [Fri Jul 1 07:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:35.926728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:35.927163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:35.927442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:35.977351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] MFA Rules not processed for user `eae0222c39364fa69be2a3cc938516bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:35.979055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:36.041896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a130ee-6e54-4f01-ac7e-1f245c2d2fd9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:36.042180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a130ee-6e54-4f01-ac7e-1f245c2d2fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:36.042451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a130ee-6e54-4f01-ac7e-1f245c2d2fd9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:36.043837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 699/1405] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 07:01:36.048646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:36.048931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:36.049209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:36.051495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 707/1406] 10.222.0.48 () {58 vars in 1039 bytes} [Fri Jul 1 07:01:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:01:36.061360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:36.067884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:36.069095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:36.070638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4073d98-0a7a-4f71-af63-d84af4228ce1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:36.076005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:36.076861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:36.077149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:36.077342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 700/1407] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:01:36.077448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:36.085706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:36.087376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b730500-9957-414d-97f1-93189f8bb497 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:36.091899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:36.099059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 708/1408] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:01:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 07:01:36.122069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:01:36.135609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35d680-6a38-4e91-bf7a-b8c609744da6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:01:36.141028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:01:36.141758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:36.141966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:36.142258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:36.151251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:01:36.152841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d45c68-7a6f-4acf-837b-c8d16d458cab service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:01:36.163017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 701/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:01:46.158236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=69 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:01:58.195225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db408-f72d-41fb-9169-009ad1e594c7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:58.195777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db408-f72d-41fb-9169-009ad1e594c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:58.195777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23db408-f72d-41fb-9169-009ad1e594c7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:58.197179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 709/1410] 10.222.0.48 () {54 vars in 918 bytes} [Fri Jul 1 07:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:58.202203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea87f-e4c9-4b6f-85f5-a8e2928504f0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:58.202660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea87f-e4c9-4b6f-85f5-a8e2928504f0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:58.202789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aea87f-e4c9-4b6f-85f5-a8e2928504f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:58.221038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-21aea87f-e4c9-4b6f-85f5-a8e2928504f0 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:58.222236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21aea87f-e4c9-4b6f-85f5-a8e2928504f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:58.231328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 702/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Fri Jul 1 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:01:58.245422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28ba42-cbd5-4f2f-bc4f-fe7cb4a9e4ed None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:58.245712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28ba42-cbd5-4f2f-bc4f-fe7cb4a9e4ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:58.245995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28ba42-cbd5-4f2f-bc4f-fe7cb4a9e4ed None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:58.247424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 710/1412] 10.222.0.48 () {54 vars in 918 bytes} [Fri Jul 1 07:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:01:58.252035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242f7e-0b3f-4a0a-aaf6-7973cace4e34 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:01:58.252324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242f7e-0b3f-4a0a-aaf6-7973cace4e34 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:01:58.252605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242f7e-0b3f-4a0a-aaf6-7973cace4e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:01:58.269802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4a242f7e-0b3f-4a0a-aaf6-7973cace4e34 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:01:58.271009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4a242f7e-0b3f-4a0a-aaf6-7973cace4e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:01:58.280457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 703/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Fri Jul 1 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:02:00.770824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17051c-51d8-4854-8b70-c41bccab5e60 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:00.771362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17051c-51d8-4854-8b70-c41bccab5e60 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:00.771362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17051c-51d8-4854-8b70-c41bccab5e60 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:00.772737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:00.777814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea466-c65d-44ed-8646-b0e24655c2c8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:00.778258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea466-c65d-44ed-8646-b0e24655c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:00.778393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefea466-c65d-44ed-8646-b0e24655c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:00.796068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aefea466-c65d-44ed-8646-b0e24655c2c8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:00.797249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aefea466-c65d-44ed-8646-b0e24655c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:00.804809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:00.847041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17051c-51d8-4854-8b70-c41bccab5e60 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:00.862172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17051c-51d8-4854-8b70-c41bccab5e60 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:00.867425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:00.868701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:00.868982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:00.869259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:00.877181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:00.878746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d7369-6d6a-46a6-8669-baadbef2214f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:00.887832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 712/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:01.785269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927de0-8d55-4a45-bb4f-8f5ec6425dfd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:01.785835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927de0-8d55-4a45-bb4f-8f5ec6425dfd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:01.785951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85927de0-8d55-4a45-bb4f-8f5ec6425dfd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:01.787369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:01.792290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d064731-01a5-43df-a020-014a1555588e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:01.792691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d064731-01a5-43df-a020-014a1555588e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:01.792850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d064731-01a5-43df-a020-014a1555588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:01.811086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9d064731-01a5-43df-a020-014a1555588e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:01.812277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d064731-01a5-43df-a020-014a1555588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:01.819933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:01.970880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85927de0-8d55-4a45-bb4f-8f5ec6425dfd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:01.985143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85927de0-8d55-4a45-bb4f-8f5ec6425dfd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:01.990479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4347968-2505-451d-882b-23427388f01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:01.991148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347968-2505-451d-882b-23427388f01f service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:01.991435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347968-2505-451d-882b-23427388f01f service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:01.991713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347968-2505-451d-882b-23427388f01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:02.000229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4347968-2505-451d-882b-23427388f01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:02.001810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4347968-2505-451d-882b-23427388f01f service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:02.011966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 706/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:02.885113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec5996-d08e-42fe-bf56-894f37d9b63e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:02.885680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec5996-d08e-42fe-bf56-894f37d9b63e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:02.885680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec5996-d08e-42fe-bf56-894f37d9b63e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:02.887114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:02.892265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25715d1c-14ff-4266-bd59-8f8c7309a1ae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:02.892668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25715d1c-14ff-4266-bd59-8f8c7309a1ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:02.892801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25715d1c-14ff-4266-bd59-8f8c7309a1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:02.918934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-25715d1c-14ff-4266-bd59-8f8c7309a1ae None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:02.920215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-25715d1c-14ff-4266-bd59-8f8c7309a1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:02.928076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:03.078245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec5996-d08e-42fe-bf56-894f37d9b63e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:03.092261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec5996-d08e-42fe-bf56-894f37d9b63e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:03.097610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:03.098240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:03.098525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:03.098837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:03.107277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:03.108849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a999326-b4f1-4b5b-ab7f-bc36cc5ce7b8 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:03.118643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 715/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:03.984885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a060c9-096d-42c0-9aee-0e10cadac51a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:03.985457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a060c9-096d-42c0-9aee-0e10cadac51a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:03.985457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a060c9-096d-42c0-9aee-0e10cadac51a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:03.986873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:03.991731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1c6d-0a83-4a68-afb1-f6c72c637ef2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:03.992169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1c6d-0a83-4a68-afb1-f6c72c637ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:03.992328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda1c6d-0a83-4a68-afb1-f6c72c637ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:04.010758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9eda1c6d-0a83-4a68-afb1-f6c72c637ef2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:04.011988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9eda1c6d-0a83-4a68-afb1-f6c72c637ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:04.019791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:04.168468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a060c9-096d-42c0-9aee-0e10cadac51a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:04.182870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a060c9-096d-42c0-9aee-0e10cadac51a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:04.188465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:04.189096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:04.189375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:04.189648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:04.198562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:04.200194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45daa7-8375-4230-80ff-ea9eb4a3ce91 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:04.209774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 709/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:05.063168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ac1e9-d5c8-4f1f-aa5e-fe8340abd8b8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:05.063748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ac1e9-d5c8-4f1f-aa5e-fe8340abd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:05.063748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ac1e9-d5c8-4f1f-aa5e-fe8340abd8b8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:05.065088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:05.069960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b715-4aa4-43f5-9121-2b66b54b5163 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:05.070354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b715-4aa4-43f5-9121-2b66b54b5163 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:05.070514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b715-4aa4-43f5-9121-2b66b54b5163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:05.088536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-13e5b715-4aa4-43f5-9121-2b66b54b5163 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:05.089733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-13e5b715-4aa4-43f5-9121-2b66b54b5163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:05.097437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:05.248266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ac1e9-d5c8-4f1f-aa5e-fe8340abd8b8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:05.262819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ac1e9-d5c8-4f1f-aa5e-fe8340abd8b8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:05.268200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:05.268839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:05.269123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:05.269401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:05.277707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:05.279334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a584d81-6d2c-4ace-88ec-134e5fc3d55a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:05.288552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 718/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:06.084930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fa586-2cf5-4bb7-9e88-d54da50685fa None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:06.085401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fa586-2cf5-4bb7-9e88-d54da50685fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:06.085510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fa586-2cf5-4bb7-9e88-d54da50685fa None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:06.087562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:06.092410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3b22-8a16-400d-bf32-9c2a25da2a69 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:06.092874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3b22-8a16-400d-bf32-9c2a25da2a69 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:06.092986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f3b22-8a16-400d-bf32-9c2a25da2a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:06.110401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a91f3b22-8a16-400d-bf32-9c2a25da2a69 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:06.111622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a91f3b22-8a16-400d-bf32-9c2a25da2a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:06.119407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:06.267891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fa586-2cf5-4bb7-9e88-d54da50685fa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:06.282915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fa586-2cf5-4bb7-9e88-d54da50685fa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:06.288223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:06.288850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:06.289134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:06.289413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:06.297457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:06.299006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a4e1f-a8d7-4cb1-9b56-c60c686280ed service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:06.308901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 712/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:07.102620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62992caf-0b34-4d16-915e-6b665618dd36 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:07.103225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62992caf-0b34-4d16-915e-6b665618dd36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:07.103225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62992caf-0b34-4d16-915e-6b665618dd36 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:07.104578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:07.109536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d210193-f34f-4bfa-a984-20b53575aac6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:07.109936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d210193-f34f-4bfa-a984-20b53575aac6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:07.110089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d210193-f34f-4bfa-a984-20b53575aac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:07.127539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7d210193-f34f-4bfa-a984-20b53575aac6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:07.128717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d210193-f34f-4bfa-a984-20b53575aac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:07.136511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:07.290717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62992caf-0b34-4d16-915e-6b665618dd36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:07.305293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62992caf-0b34-4d16-915e-6b665618dd36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:07.310953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:07.311584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:07.311868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:07.312145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:07.320106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:07.321721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e3dbf-86a4-4c8c-939c-a9944dbdbd69 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:07.330990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 721/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:02:08.128901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c724-c045-4f98-bab4-b3979cf3d9fe None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:08.129384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c724-c045-4f98-bab4-b3979cf3d9fe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:08.129492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c724-c045-4f98-bab4-b3979cf3d9fe None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:08.130875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:08.136308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58006b-34ec-40b8-b21d-4f91b1599eb0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:08.136703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58006b-34ec-40b8-b21d-4f91b1599eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:08.136868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58006b-34ec-40b8-b21d-4f91b1599eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:08.154336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7e58006b-34ec-40b8-b21d-4f91b1599eb0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:08.155559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7e58006b-34ec-40b8-b21d-4f91b1599eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:08.163285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:08.312622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111c724-c045-4f98-bab4-b3979cf3d9fe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:08.326391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111c724-c045-4f98-bab4-b3979cf3d9fe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:08.331650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:08.332273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:08.332551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:08.332853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:08.340776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:08.342336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9815b5a-0435-4794-a07e-0c97a8f85fc1 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:08.351408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 715/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:09.160741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4b7a-e54e-4687-a10e-9e012038f935 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:09.161306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4b7a-e54e-4687-a10e-9e012038f935 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:09.161306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce4b7a-e54e-4687-a10e-9e012038f935 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:09.162671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:09.167656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38804b4f-5ff8-492c-8c47-f1e92f62c86d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:09.168109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38804b4f-5ff8-492c-8c47-f1e92f62c86d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:09.168220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38804b4f-5ff8-492c-8c47-f1e92f62c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:09.186261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-38804b4f-5ff8-492c-8c47-f1e92f62c86d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:09.187504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38804b4f-5ff8-492c-8c47-f1e92f62c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:09.195404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:09.345326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce4b7a-e54e-4687-a10e-9e012038f935 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:09.359674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce4b7a-e54e-4687-a10e-9e012038f935 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:09.365060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:09.365691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:09.365978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:09.366259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:09.374910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:09.376501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20690f2f-a142-4106-9217-f41797b60cf7 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:09.386407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 724/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:11.293732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3496-6da9-44f0-a6d5-00263fb142d1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:11.294264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3496-6da9-44f0-a6d5-00263fb142d1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:11.294264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc3496-6da9-44f0-a6d5-00263fb142d1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:11.295659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:11.300766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f9a6-6b13-41b8-98f2-24716407fe9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:11.301196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f9a6-6b13-41b8-98f2-24716407fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:11.301307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f9a6-6b13-41b8-98f2-24716407fe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:11.320727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4543f9a6-6b13-41b8-98f2-24716407fe9f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:11.321929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4543f9a6-6b13-41b8-98f2-24716407fe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:11.329665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:11.368513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3496-6da9-44f0-a6d5-00263fb142d1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:11.379269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc3496-6da9-44f0-a6d5-00263fb142d1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:11.382689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:11.383368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:11.383647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:11.383922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:11.392555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:11.393962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a018d3-9bc4-4fcd-99e5-3c28826f8772 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:11.403182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 718/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:02:13.551510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706853e6-6caf-41eb-b147-3c4f3bf6b657 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:13.552043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706853e6-6caf-41eb-b147-3c4f3bf6b657 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:13.552043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706853e6-6caf-41eb-b147-3c4f3bf6b657 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:13.553342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:13.558640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41cfec-b8c7-4f8d-9a9c-b8491e20ce61 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:13.559107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41cfec-b8c7-4f8d-9a9c-b8491e20ce61 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:13.559241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41cfec-b8c7-4f8d-9a9c-b8491e20ce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:13.578005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6f41cfec-b8c7-4f8d-9a9c-b8491e20ce61 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:13.579240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f41cfec-b8c7-4f8d-9a9c-b8491e20ce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:13.587863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:13.624873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706853e6-6caf-41eb-b147-3c4f3bf6b657 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:13.636261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706853e6-6caf-41eb-b147-3c4f3bf6b657 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:13.639847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:13.640475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:13.640758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:13.641036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:13.650845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:13.652313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b365ef-2846-445d-851b-34e3d86c406a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:13.661653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 727/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:15.954620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215a81a-d605-48e6-b190-88dd7dc958ae None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:15.955220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215a81a-d605-48e6-b190-88dd7dc958ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:15.955220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215a81a-d605-48e6-b190-88dd7dc958ae None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:15.956573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:15.961654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca3b4-a916-496f-a7dc-5bfc715aa3cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:15.962098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca3b4-a916-496f-a7dc-5bfc715aa3cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:15.962207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ca3b4-a916-496f-a7dc-5bfc715aa3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:15.981834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-068ca3b4-a916-496f-a7dc-5bfc715aa3cb None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:15.983063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-068ca3b4-a916-496f-a7dc-5bfc715aa3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:15.990817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:16.030052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215a81a-d605-48e6-b190-88dd7dc958ae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:16.041153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215a81a-d605-48e6-b190-88dd7dc958ae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:16.044590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:16.045209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:16.045486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:16.045756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:16.054574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:16.056033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942b8a2-c162-4694-a3f4-3de8416a77ad service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:16.065508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 721/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:18.221835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd220f-9052-4ee4-99fd-c6e7c695ec4f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:18.222235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd220f-9052-4ee4-99fd-c6e7c695ec4f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:18.222392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd220f-9052-4ee4-99fd-c6e7c695ec4f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:18.223849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:18.229217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc2b5-b79f-4d16-aad2-027977694dad None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:18.229638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc2b5-b79f-4d16-aad2-027977694dad None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:18.229827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fc2b5-b79f-4d16-aad2-027977694dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:18.247243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-244fc2b5-b79f-4d16-aad2-027977694dad None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:18.248459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-244fc2b5-b79f-4d16-aad2-027977694dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:18.256344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:18.295248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd220f-9052-4ee4-99fd-c6e7c695ec4f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:18.306912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd220f-9052-4ee4-99fd-c6e7c695ec4f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:18.310582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:18.311251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:18.311527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:18.311797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:18.321636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:18.323092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df6b5b9-0234-4931-a0d1-2ac7f46c35c5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:18.332182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 730/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:20.417178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be9e-0ef5-474f-93cc-c96175f18eec None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:20.417745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be9e-0ef5-474f-93cc-c96175f18eec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:20.417745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660be9e-0ef5-474f-93cc-c96175f18eec None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:20.419124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:20.424257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa45042-e749-47ef-9496-7b094ee288be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:20.424717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa45042-e749-47ef-9496-7b094ee288be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:20.424829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa45042-e749-47ef-9496-7b094ee288be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:20.448123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8fa45042-e749-47ef-9496-7b094ee288be None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:20.449327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8fa45042-e749-47ef-9496-7b094ee288be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:20.457187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:20.495004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660be9e-0ef5-474f-93cc-c96175f18eec None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:20.505858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660be9e-0ef5-474f-93cc-c96175f18eec None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:20.509938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:20.510578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:20.510899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:20.511181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:20.519946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:20.521369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c4f46-814d-4506-816e-c592b24414c0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:20.531311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 724/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:22.607665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba16d2-5f71-4856-8fcd-2c159c06c003 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:22.608232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba16d2-5f71-4856-8fcd-2c159c06c003 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:22.608232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba16d2-5f71-4856-8fcd-2c159c06c003 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:22.609637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:22.614863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328591-bb22-4990-9bae-d5390a6bca9e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:22.615293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328591-bb22-4990-9bae-d5390a6bca9e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:22.615408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a328591-bb22-4990-9bae-d5390a6bca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:22.633546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9a328591-bb22-4990-9bae-d5390a6bca9e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:22.634740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9a328591-bb22-4990-9bae-d5390a6bca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:22.642450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:22.680423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba16d2-5f71-4856-8fcd-2c159c06c003 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:22.691428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba16d2-5f71-4856-8fcd-2c159c06c003 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:22.694917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:22.695572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:22.695855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:22.696131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:22.704908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:22.706349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6499fed7-2e4c-4341-9189-f8d9851d9748 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:22.715850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 733/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:24.780887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913be509-7fe7-4eb6-9f23-40d85144e7e5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:24.781463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913be509-7fe7-4eb6-9f23-40d85144e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:24.781463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913be509-7fe7-4eb6-9f23-40d85144e7e5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:24.782856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:24.787925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551d75-11bf-4967-99a6-8542960949f5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:24.788367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551d75-11bf-4967-99a6-8542960949f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:24.788504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41551d75-11bf-4967-99a6-8542960949f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:24.805545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41551d75-11bf-4967-99a6-8542960949f5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:24.806721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41551d75-11bf-4967-99a6-8542960949f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:24.814616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:24.852046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913be509-7fe7-4eb6-9f23-40d85144e7e5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:24.862740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913be509-7fe7-4eb6-9f23-40d85144e7e5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:24.866280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:24.866956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:24.867242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:24.867520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:24.876406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:24.877814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cda40-4a77-4d23-870f-3005639d9a70 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:24.887624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 727/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:26.975079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd09c0-a0d6-47a8-bc6c-a9462f63cfb0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:26.975644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd09c0-a0d6-47a8-bc6c-a9462f63cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:26.975644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd09c0-a0d6-47a8-bc6c-a9462f63cfb0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:26.977003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:26.982163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da5a2-635a-4726-96ef-cb865e9921b2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:26.982543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da5a2-635a-4726-96ef-cb865e9921b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:26.982732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70da5a2-635a-4726-96ef-cb865e9921b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:26.988756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=79 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:02:27.001251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f70da5a2-635a-4726-96ef-cb865e9921b2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:27.002456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f70da5a2-635a-4726-96ef-cb865e9921b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:27.010719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:27.048065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd09c0-a0d6-47a8-bc6c-a9462f63cfb0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:27.053371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=73 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:02:27.060019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd09c0-a0d6-47a8-bc6c-a9462f63cfb0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:27.063470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:27.064096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:27.064375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:27.064645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:27.074830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:27.077337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb2d8-190b-471f-bdc4-829a825e33d0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:27.088161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 736/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:29.153476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023c01-9497-4c36-9ae7-1bc162bb7289 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:29.154076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023c01-9497-4c36-9ae7-1bc162bb7289 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:29.154076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89023c01-9497-4c36-9ae7-1bc162bb7289 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:29.155459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:29.160667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a751a69-572c-43c9-8119-658ce221d7c4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:29.161120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a751a69-572c-43c9-8119-658ce221d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:29.161231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a751a69-572c-43c9-8119-658ce221d7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:29.180220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9a751a69-572c-43c9-8119-658ce221d7c4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:29.181412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9a751a69-572c-43c9-8119-658ce221d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:29.189345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:29.225218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89023c01-9497-4c36-9ae7-1bc162bb7289 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:29.236466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89023c01-9497-4c36-9ae7-1bc162bb7289 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:29.239970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:29.240601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:29.240886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:29.241164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:29.250558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:29.252030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0695b-4c91-4a3e-a8b1-033639277a25 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:29.261798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 730/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:31.285682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa568238-b6a1-4933-866c-a2e11f4369b2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:31.286186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa568238-b6a1-4933-866c-a2e11f4369b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:31.286281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa568238-b6a1-4933-866c-a2e11f4369b2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:31.287622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:31.292985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2ea5f-50e6-4f60-8edd-e99d375c2521 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:31.293399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2ea5f-50e6-4f60-8edd-e99d375c2521 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:31.293536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2ea5f-50e6-4f60-8edd-e99d375c2521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:31.311477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1ef2ea5f-50e6-4f60-8edd-e99d375c2521 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:31.312705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ef2ea5f-50e6-4f60-8edd-e99d375c2521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:31.322138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:31.357364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa568238-b6a1-4933-866c-a2e11f4369b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:31.368417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa568238-b6a1-4933-866c-a2e11f4369b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:31.371872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:31.372505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:31.372788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:31.373066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:31.382186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:31.384218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd72e7b-cfd7-44f6-b56e-ffc613625ee1 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:31.393788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 739/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:41.306256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=11 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:02:41.389706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=14 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:02:51.224402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fc93-1555-4b69-9948-0b2ce31a641a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:51.224979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fc93-1555-4b69-9948-0b2ce31a641a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:51.224979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fc93-1555-4b69-9948-0b2ce31a641a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:51.226840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 732/1471] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:51.232467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0bfd-24ff-4a14-a6f1-fe6fd28fa9ed None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:51.232910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0bfd-24ff-4a14-a6f1-fe6fd28fa9ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:51.233028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce0bfd-24ff-4a14-a6f1-fe6fd28fa9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:51.250835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-02ce0bfd-24ff-4a14-a6f1-fe6fd28fa9ed None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:51.252034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-02ce0bfd-24ff-4a14-a6f1-fe6fd28fa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:51.259728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 740/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:02:51.307153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb472bd-24f8-41f7-8813-8b9faa85093a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:51.307449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb472bd-24f8-41f7-8813-8b9faa85093a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:51.307734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb472bd-24f8-41f7-8813-8b9faa85093a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:51.309074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 733/1473] 10.222.0.48 () {54 vars in 962 bytes} [Fri Jul 1 07:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:51.314001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c4a2-9626-49c1-ad05-971b6df061df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:51.314289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c4a2-9626-49c1-ad05-971b6df061df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:51.314567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11c4a2-9626-49c1-ad05-971b6df061df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:51.334023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ff11c4a2-9626-49c1-ad05-971b6df061df None None] MFA Rules not processed for user `f1d06e6ad60240329d83bc4767439ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:51.335746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff11c4a2-9626-49c1-ad05-971b6df061df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:51.342118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 741/1474] 10.222.0.48 () {58 vars in 1059 bytes} [Fri Jul 1 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:51.349683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb472bd-24f8-41f7-8813-8b9faa85093a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:51.360786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb472bd-24f8-41f7-8813-8b9faa85093a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:51.364291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:51.364918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:51.365204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:51.365482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:51.374291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:51.375752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f57a6-df03-4abc-8fb1-4c7b787c1167 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:51.385046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 734/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:53.487008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143701-10a3-4b20-b21b-6ac813b97aeb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:53.487527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143701-10a3-4b20-b21b-6ac813b97aeb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:53.487527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40143701-10a3-4b20-b21b-6ac813b97aeb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:53.488852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 742/1476] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:53.493948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139b1e-5d99-4ba0-b218-d604844f851e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:53.494350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139b1e-5d99-4ba0-b218-d604844f851e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:53.494514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4139b1e-5d99-4ba0-b218-d604844f851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:53.512547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a4139b1e-5d99-4ba0-b218-d604844f851e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:53.513745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a4139b1e-5d99-4ba0-b218-d604844f851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:53.521544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 735/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:53.544725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143701-10a3-4b20-b21b-6ac813b97aeb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:53.557626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40143701-10a3-4b20-b21b-6ac813b97aeb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:53.562276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:53.562951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:53.563232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:53.563504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:53.564268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:53.566046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:02:53.566046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:02:53.566046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:02:53.567247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3c28c-780d-486e-9ac6-532f8c23da11 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:53.586865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 743/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Fri Jul 1 07:02:53 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:02:55.630441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd76e9-2cd0-4b92-ade1-8fd3ec4b6996 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:55.631035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd76e9-2cd0-4b92-ade1-8fd3ec4b6996 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:55.631035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd76e9-2cd0-4b92-ade1-8fd3ec4b6996 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:55.632524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 736/1479] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:55.637994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f537b-fbf8-4fd8-9da2-9885c2e11fe1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:55.638463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f537b-fbf8-4fd8-9da2-9885c2e11fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:55.638602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f537b-fbf8-4fd8-9da2-9885c2e11fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:55.656270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c1f537b-fbf8-4fd8-9da2-9885c2e11fe1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:55.657463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c1f537b-fbf8-4fd8-9da2-9885c2e11fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:55.665731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 744/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:02:55.709201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd76e9-2cd0-4b92-ade1-8fd3ec4b6996 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:55.723608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd76e9-2cd0-4b92-ade1-8fd3ec4b6996 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:55.729454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:55.730086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:55.730370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:55.730647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:55.738641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:55.740249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436d76a-70e4-4e0a-acc4-1a22cc1144ed service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:55.750096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 737/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:02:57.949831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93b1c-4a32-4f0c-979b-bb6542b619a8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:57.950403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93b1c-4a32-4f0c-979b-bb6542b619a8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:57.950403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f93b1c-4a32-4f0c-979b-bb6542b619a8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:57.951794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 745/1482] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:02:57.957053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01413-7361-44ab-867b-478cd81a3153 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:57.957471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01413-7361-44ab-867b-478cd81a3153 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:57.957613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c01413-7361-44ab-867b-478cd81a3153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:57.976060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d5c01413-7361-44ab-867b-478cd81a3153 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:02:57.977257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5c01413-7361-44ab-867b-478cd81a3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:02:57.984984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 738/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:02:58.020749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f93b1c-4a32-4f0c-979b-bb6542b619a8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:02:58.032004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f93b1c-4a32-4f0c-979b-bb6542b619a8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:02:58.035557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:02:58.036215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:02:58.036522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:02:58.036821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:02:58.045706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:02:58.047170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df98751-6375-4767-8bd4-2dcaa5e2a1c0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:02:58.056930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 746/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:03:00.126908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb852b1-7b72-4d99-8aad-814a904c47be None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:00.127397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb852b1-7b72-4d99-8aad-814a904c47be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:00.127517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb852b1-7b72-4d99-8aad-814a904c47be None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:00.128838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 739/1485] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:03:00.134097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62546b-fb8a-4f8a-842c-4cc7a96416ce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:00.134529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62546b-fb8a-4f8a-842c-4cc7a96416ce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:00.134653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62546b-fb8a-4f8a-842c-4cc7a96416ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:00.151801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd62546b-fb8a-4f8a-842c-4cc7a96416ce None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:03:00.152980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd62546b-fb8a-4f8a-842c-4cc7a96416ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:03:00.160614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 747/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:03:00.198868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb852b1-7b72-4d99-8aad-814a904c47be None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:03:00.209527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb852b1-7b72-4d99-8aad-814a904c47be None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:03:00.213136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:03:00.213773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:00.214058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:00.214338 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:00.223355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:03:00.224782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd7f21-eb19-47d8-9314-3e27efbaff64 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:03:00.234602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 740/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:03:02.356953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874306-d154-435f-b1e9-2571f40f599d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:02.357491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874306-d154-435f-b1e9-2571f40f599d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:02.357491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70874306-d154-435f-b1e9-2571f40f599d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:02.358908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 748/1488] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:03:02.364119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5339-4fc6-4a94-b140-c3b74b5f9a2d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:02.364549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5339-4fc6-4a94-b140-c3b74b5f9a2d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:02.364672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b5339-4fc6-4a94-b140-c3b74b5f9a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:02.382240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-243b5339-4fc6-4a94-b140-c3b74b5f9a2d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:03:02.383459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-243b5339-4fc6-4a94-b140-c3b74b5f9a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:03:02.391074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 741/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:03:02.430368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70874306-d154-435f-b1e9-2571f40f599d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:03:02.441534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70874306-d154-435f-b1e9-2571f40f599d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:03:02.445676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:03:02.446304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:02.446589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:02.446905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:02.455676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:03:02.457095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d9010-e885-4a77-af6f-2ce83293b2a1 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:03:02.466959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 749/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:03:04.660758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4c1f-2521-4307-8a96-3f118bfb385a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:04.661302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4c1f-2521-4307-8a96-3f118bfb385a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:04.661403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4c1f-2521-4307-8a96-3f118bfb385a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:04.662721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 742/1491] 10.222.0.48 () {54 vars in 926 bytes} [Fri Jul 1 07:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:03:04.668069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f209-9372-43a6-bf17-c73381a34580 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:04.668508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f209-9372-43a6-bf17-c73381a34580 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:04.668615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f209-9372-43a6-bf17-c73381a34580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:04.687860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1f73f209-9372-43a6-bf17-c73381a34580 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:03:04.689061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1f73f209-9372-43a6-bf17-c73381a34580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:03:04.697303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 750/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Fri Jul 1 07:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:03:04.736151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c4c1f-2521-4307-8a96-3f118bfb385a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:03:04.746958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c4c1f-2521-4307-8a96-3f118bfb385a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:03:04.750385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:03:04.751053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:03:04.751355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:03:04.751629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:03:04.760752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:03:04.762164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038ecce-3081-4e6b-87b6-7632a1c1574b service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:03:04.771350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 743/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:03:14.682672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=33 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:03:14.767193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=33 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:04:31.562762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.563372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.563372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.580808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:04:31.581992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:31.590149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 751/1494] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:04:31.598010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.599096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.599207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.616918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:04:31.618566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:31.627716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 744/1495] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:04:31.632344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.641925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0ab0-16e0-4252-9e52-0dc18fd93db3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.646891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.647531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.647813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.648079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.648789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:04:31.649091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.650091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.650091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:04:31.650091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:04:31.651034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.662712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 752/1496] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:04:31.670615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.679702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690bdb-1d01-40db-b111-2867324caa3c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.684258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.685167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.685453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.685730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.686421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.688210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.688210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:04:31.688210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:04:31.689396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.724804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 745/1497] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:04:31 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:04:31.728978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.738468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd12731-0fb3-4d20-89fa-97f15a1258c4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.792625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.793276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.793581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.793825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.794506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.795592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.795592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:04:31.795592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:04:31.796781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.815749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 753/1498] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:04:31 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:04:31.820374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.830388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fb426-2586-4bd9-a414-140c7be88bea admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.835539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.836188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.836472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.836748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.837469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.838474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.838474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:04:31.838474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:04:31.839394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.851924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 746/1499] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:04:31.855581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.865046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b054b5d7-ad4d-49f3-a4f6-4bf9e0b3a344 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.869466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.870141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.870421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.870695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.871439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.872433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.872433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:04:31.872433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:04:31.873295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3cbab-c70a-42c0-9e5e-91e761f0da99 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.886110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 754/1500] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:04:31.889827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:31.900088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e2567-959f-4f5f-bfb9-a3e3de6b5422 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:31.904641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:31.905498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.905783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.906060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] PATH_INFO: `/v3/projects/166cefd232d74f65903404b566871ab4/users/413db6c24a9b46b9b6dc2c8e81ce8748/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.920298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=166cefd232d74f65903404b566871ab4, user_id=413db6c24a9b46b9b6dc2c8e81ce8748, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:31.921315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:31.921315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:04:31.921315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:04:31.923281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52ffb2-363f-46f4-a2be-75ea34b63b9a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:31.935903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 747/1501] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:04:31 2022] PUT /identity/v3/projects/166cefd232d74f65903404b566871ab4/users/413db6c24a9b46b9b6dc2c8e81ce8748/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:04:31.948718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7936-8e9a-4af3-85db-b2f81d17a8d9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:31.949011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7936-8e9a-4af3-85db-b2f81d17a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:31.949290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db7936-8e9a-4af3-85db-b2f81d17a8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:31.975192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b9db7936-8e9a-4af3-85db-b2f81d17a8d9 None None] MFA Rules not processed for user `413db6c24a9b46b9b6dc2c8e81ce8748`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:04:31.976420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b9db7936-8e9a-4af3-85db-b2f81d17a8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:32.037605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 755/1502] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:04:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:04:32.050444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d07ba-5b81-437a-861e-902708dafec6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.050736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d07ba-5b81-437a-861e-902708dafec6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.051072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d07ba-5b81-437a-861e-902708dafec6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.052465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 748/1503] 10.222.0.48 () {54 vars in 963 bytes} [Fri Jul 1 07:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:04:32.057376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff04236-7aaa-4877-a8cc-15ad3b0e2d37 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.057672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff04236-7aaa-4877-a8cc-15ad3b0e2d37 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.057957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff04236-7aaa-4877-a8cc-15ad3b0e2d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.077243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ff04236-7aaa-4877-a8cc-15ad3b0e2d37 None None] MFA Rules not processed for user `52d7bcc4b67c495f821519996cc2ab26`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:04:32.078375 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ff04236-7aaa-4877-a8cc-15ad3b0e2d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:32.119950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:04:32.128197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 756/1504] 10.222.0.48 () {58 vars in 1060 bytes} [Fri Jul 1 07:04:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:04:32.136458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d07ba-5b81-437a-861e-902708dafec6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:32.147480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d07ba-5b81-437a-861e-902708dafec6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:32.150904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:32.151530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.151806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.152078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.160104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:32.161516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:32.170081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 749/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:04:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:04:32.212491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870451-1fc2-4902-aa9e-75b3a1729744 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.212845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870451-1fc2-4902-aa9e-75b3a1729744 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.213124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2870451-1fc2-4902-aa9e-75b3a1729744 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.214420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 757/1506] 10.222.0.48 () {52 vars in 829 bytes} [Fri Jul 1 07:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:04:32.218045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:32.277538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572d043-47e2-4416-98dc-4c347885fde7 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:32.281970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:32.282856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.283138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.283412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] PATH_INFO: `/v3/users/413db6c24a9b46b9b6dc2c8e81ce8748` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.284578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=413db6c24a9b46b9b6dc2c8e81ce8748)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:32.285606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:32.285606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:04:32.285606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:04:32.286832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:32.349597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413db6c24a9b46b9b6dc2c8e81ce8748'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:32.358692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413db6c24a9b46b9b6dc2c8e81ce8748'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:32.359348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413db6c24a9b46b9b6dc2c8e81ce8748'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:32.377141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413db6c24a9b46b9b6dc2c8e81ce8748'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:32.377865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413db6c24a9b46b9b6dc2c8e81ce8748'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:32.386971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:04:32.398704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 750/1507] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:04:32 2022] DELETE /identity/v3/users/413db6c24a9b46b9b6dc2c8e81ce8748 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:04:32.403079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403802e-281a-486e-8840-fda989e798d7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.403365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403802e-281a-486e-8840-fda989e798d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.403660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7403802e-281a-486e-8840-fda989e798d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.420561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7403802e-281a-486e-8840-fda989e798d7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:04:32.421758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7403802e-281a-486e-8840-fda989e798d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:32.477935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 758/1508] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:04:32.487279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:32.489284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:32.545459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c33ec-eddd-4d99-a994-bba5ead8a29f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:32.549303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:32.549949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:32.550234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:32.550510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:32.560269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:32.561695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43dbb9-ade6-48a1-b932-df6f4e7aa1a9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:32.571624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 751/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:04:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:04:33.031558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403802e-281a-486e-8840-fda989e798d7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:04:33.033052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7403802e-281a-486e-8840-fda989e798d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:04:33.046062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7403802e-281a-486e-8840-fda989e798d7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:04:33.050707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:04:33.051672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:04:33.051959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:04:33.052400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] PATH_INFO: `/v3/projects/166cefd232d74f65903404b566871ab4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:04:33.053614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=166cefd232d74f65903404b566871ab4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:04:33.054642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:04:33.054642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:04:33.054642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:04:33.055925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:04:33.117539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166cefd232d74f65903404b566871ab4', 'request_id': 'req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:33.131312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166cefd232d74f65903404b566871ab4', 'request_id': 'req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:33.141132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166cefd232d74f65903404b566871ab4', 'request_id': 'req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:33.152699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] The token cache is being invalidate because project 166cefd232d74f65903404b566871ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:04:33.152981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:33.153688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-22bd2584-9adc-4f54-a83c-aa13e0eeb5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:04:33.155271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 759/1510] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:04:33 2022] DELETE /identity/v3/projects/166cefd232d74f65903404b566871ab4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:04:42.566969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=29 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:04:43.149661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:06:15.011418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:15.012009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:15.012009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:15.013453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:15.018953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959097-a1de-4def-9630-af2b2cadbbb6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:15.019404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959097-a1de-4def-9630-af2b2cadbbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:15.019513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959097-a1de-4def-9630-af2b2cadbbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:15.036912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-96959097-a1de-4def-9630-af2b2cadbbb6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:15.038107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-96959097-a1de-4def-9630-af2b2cadbbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:15.094583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:15.246150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:15.247863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:15.322199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b055f-46be-4b88-824f-76c6f3bc1f31 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:15.327971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:15.328610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:15.328898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:15.329182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:15.337208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:15.338816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2d81a-3323-4af0-bcd7-eba0435bfde0 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:15.348352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 753/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:16.172167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af824-bfcd-40ae-814a-5bd6d6b5e59a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:16.172735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af824-bfcd-40ae-814a-5bd6d6b5e59a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:16.172735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793af824-bfcd-40ae-814a-5bd6d6b5e59a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:16.174694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:16.179407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2ba40-f40c-430e-9aa2-433133c4d1a8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:16.179858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2ba40-f40c-430e-9aa2-433133c4d1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:16.179998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2ba40-f40c-430e-9aa2-433133c4d1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:16.199181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-35d2ba40-f40c-430e-9aa2-433133c4d1a8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:16.200405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-35d2ba40-f40c-430e-9aa2-433133c4d1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:16.208305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:16.358945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793af824-bfcd-40ae-814a-5bd6d6b5e59a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:16.373622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793af824-bfcd-40ae-814a-5bd6d6b5e59a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:16.380623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:16.381125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:16.381406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:16.381772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:16.389975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:16.391617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aeaaa-0645-4b28-9a54-7418f6ca1f51 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:16.400974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 762/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:16.883239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1350-d0c1-4649-b7da-a97f763ea93e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:16.883534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1350-d0c1-4649-b7da-a97f763ea93e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:16.883820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1350-d0c1-4649-b7da-a97f763ea93e None None] PATH_INFO: `/v3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:16.885132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 755/1517] 10.222.0.46 () {54 vars in 925 bytes} [Fri Jul 1 07:06:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:06:16.891114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954088-82a1-4f10-97bf-169808176b91 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:16.891407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954088-82a1-4f10-97bf-169808176b91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:16.891689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954088-82a1-4f10-97bf-169808176b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:16.910563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7c954088-82a1-4f10-97bf-169808176b91 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:16.911802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c954088-82a1-4f10-97bf-169808176b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:16.973197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 763/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Jul 1 07:06:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:25.979393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a5a3-8fad-460a-8dce-dcdd46488aec None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:25.979917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a5a3-8fad-460a-8dce-dcdd46488aec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:25.979917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a5a3-8fad-460a-8dce-dcdd46488aec None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:25.981216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 756/1519] 10.222.0.48 () {54 vars in 960 bytes} [Fri Jul 1 07:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:25.986305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9e81-e03f-4f8a-a3a3-c39781e5202f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:25.986752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9e81-e03f-4f8a-a3a3-c39781e5202f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:25.986900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9e81-e03f-4f8a-a3a3-c39781e5202f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:26.004239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-368c9e81-e03f-4f8a-a3a3-c39781e5202f None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:26.005843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-368c9e81-e03f-4f8a-a3a3-c39781e5202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:26.015310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 764/1520] 10.222.0.48 () {58 vars in 1057 bytes} [Fri Jul 1 07:06:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:26.023639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967a5a3-8fad-460a-8dce-dcdd46488aec None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:26.037367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967a5a3-8fad-460a-8dce-dcdd46488aec None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:26.042556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:26.043281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:26.043565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:26.043851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:26.052275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:26.053875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e1a8-0dd9-4f87-82a8-c8691bf24466 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:26.063738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 757/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:27.626212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6e17c-3e57-4996-b27c-b85050aa44a0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:27.626734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6e17c-3e57-4996-b27c-b85050aa44a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:27.626871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6e17c-3e57-4996-b27c-b85050aa44a0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:27.628788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:27.633567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7915e2c-c8bb-4b88-a28c-16aa7b2068d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:27.633991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7915e2c-c8bb-4b88-a28c-16aa7b2068d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:27.634099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7915e2c-c8bb-4b88-a28c-16aa7b2068d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:27.652236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a7915e2c-c8bb-4b88-a28c-16aa7b2068d8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:27.653451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a7915e2c-c8bb-4b88-a28c-16aa7b2068d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:27.661336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:27.814665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6e17c-3e57-4996-b27c-b85050aa44a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:27.829773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6e17c-3e57-4996-b27c-b85050aa44a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:27.835624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:27.836250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:27.836532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:27.836807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:27.845043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:27.846675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cdce83-c7d2-4b9f-9b89-bb658db1ea97 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:27.855862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 766/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:28.677491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc80d95-a9b9-49ab-978c-49c6ebcc05be None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:28.678068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc80d95-a9b9-49ab-978c-49c6ebcc05be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:28.678068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc80d95-a9b9-49ab-978c-49c6ebcc05be None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:28.679441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 1 07:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:28.684142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778726-4990-43b8-843f-f087f0e4998f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:28.684850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778726-4990-43b8-843f-f087f0e4998f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:28.684955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd778726-4990-43b8-843f-f087f0e4998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:28.703951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dd778726-4990-43b8-843f-f087f0e4998f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:28.705141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd778726-4990-43b8-843f-f087f0e4998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:28.712910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 1 07:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:28.862631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc80d95-a9b9-49ab-978c-49c6ebcc05be None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:28.876704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc80d95-a9b9-49ab-978c-49c6ebcc05be None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:28.882003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:28.882626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:28.882948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:28.883226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:28.891199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:28.892766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb65881e-d359-491f-84d0-34de0199f094 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:28.902447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 760/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:32.835021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:32.835607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:32.835607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:32.836964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:32.842320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978d64-bdcd-4c60-bf80-41f2e720a7ef None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:32.842789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978d64-bdcd-4c60-bf80-41f2e720a7ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:32.842917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978d64-bdcd-4c60-bf80-41f2e720a7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:32.860424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8d978d64-bdcd-4c60-bf80-41f2e720a7ef None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:32.861611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d978d64-bdcd-4c60-bf80-41f2e720a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:32.869851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:32.907739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:32.909934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:32.966162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125582ce-2102-444d-aa8c-70b68da911b3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:32.969697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:32.970335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:32.970622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:32.970933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:32.980452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:32.981883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc19ff-ba99-47be-b2ec-f5ad4d39c92e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:32.987627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:06:32.992818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 769/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:35.008044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c9-c8a3-4fc2-964c-7546dbf3289e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:35.008623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c9-c8a3-4fc2-964c-7546dbf3289e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:35.008623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a2c9-c8a3-4fc2-964c-7546dbf3289e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:35.009943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:35.015069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b422-f2de-414a-9790-630efecd2a1a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:35.015456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b422-f2de-414a-9790-630efecd2a1a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:35.015627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3b422-f2de-414a-9790-630efecd2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:35.034413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-18e3b422-f2de-414a-9790-630efecd2a1a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:35.035696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-18e3b422-f2de-414a-9790-630efecd2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:35.043523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:35.079400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a2c9-c8a3-4fc2-964c-7546dbf3289e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:35.090644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a2c9-c8a3-4fc2-964c-7546dbf3289e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:35.094150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:35.094795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:35.095094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:35.095373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:35.104867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:35.106295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71cbac-026e-45ba-84c1-b3232c23a0fe service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:35.115703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 763/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:38.633864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:38.634424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:38.634424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:38.635837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:38.641032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751e1d-5f1f-40fd-9464-3cd4b5b30050 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:38.641458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751e1d-5f1f-40fd-9464-3cd4b5b30050 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:38.641576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f751e1d-5f1f-40fd-9464-3cd4b5b30050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:38.659036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f751e1d-5f1f-40fd-9464-3cd4b5b30050 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:38.660230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f751e1d-5f1f-40fd-9464-3cd4b5b30050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:38.667931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:38.970234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:38.972494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:39.027774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9a06-bfde-43bb-820b-be971678814e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:39.031278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:39.031913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:39.032198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:39.032474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:39.041726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:39.043196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98b878-640d-412b-bcc2-5551a5f7c9fb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:39.052954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 772/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:06:41.106054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:41.106625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:41.106625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:41.108080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 1 07:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:06:41.113847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31464-6ac1-4e75-8cef-ae20c813d33e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:41.114267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31464-6ac1-4e75-8cef-ae20c813d33e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:41.114408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31464-6ac1-4e75-8cef-ae20c813d33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:41.134643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-38e31464-6ac1-4e75-8cef-ae20c813d33e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:06:41.136076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-38e31464-6ac1-4e75-8cef-ae20c813d33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:41.146889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 1 07:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:06:41.451812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:06:41.453536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:06:41.466402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e92fb-64f4-4593-aef2-33e92a847989 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:06:41.469920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:06:41.470544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:06:41.470854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:06:41.471633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:06:41.480996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:06:41.482415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae0486-2169-4b51-92a8-1d0e8698a721 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:06:41.492272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 766/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:06:51.127252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:06:51.488200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=70 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:11.460702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.461399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.461399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.490560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:11.491892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:11.504201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 774/1540] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:11.512920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.513516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.513516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.543899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:11.545272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:11.558072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 767/1541] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:11.563733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:11.585811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641ed6c-357e-4f0d-8406-bde7812cac3a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:11.599408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:11.600096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.600380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.601263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.602017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:11.602326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:11.604003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:11.618756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 775/1542] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:11.627568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:11.646685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9256f10-954b-4f2a-a9a7-a82aee080d4f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:11.655250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:11.655918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.656204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.656517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.657378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:11.659356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4c2af1-c385-47f3-9f86-0c555d92806f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:11.705353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 768/1543] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/projects => generated 405 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:11.709542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:11.724452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf29ce0-55c8-4e61-83f1-b876f65c61d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:11.804284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:11.804880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.805163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.805450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.806149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:11.808127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:11.835455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85a5a-8cbc-46dc-a6af-20ef02e466bd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.835822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85a5a-8cbc-46dc-a6af-20ef02e466bd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.836087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a85a5a-8cbc-46dc-a6af-20ef02e466bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.847977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 776/1544] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:11.851915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:11.883489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c4d6-e50a-4701-93d9-057950304c94 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:11.884480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c1a85a5a-8cbc-46dc-a6af-20ef02e466bd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:11.886998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1a85a5a-8cbc-46dc-a6af-20ef02e466bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:11.897562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:11.897562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.897562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.897562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.898291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:11.902807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8da2a1-170e-493d-8db9-574907cad358 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:11.911150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 769/1545] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:11.915087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.915380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.915660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.931114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 777/1546] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:08:11.932759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f54-bb3c-4c19-a577-d2fd344d63a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.933078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f54-bb3c-4c19-a577-d2fd344d63a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.934222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c5f54-bb3c-4c19-a577-d2fd344d63a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.952107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:11.953523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:11.966757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a4c5f54-bb3c-4c19-a577-d2fd344d63a4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:11.968035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a4c5f54-bb3c-4c19-a577-d2fd344d63a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:11.968510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 770/1547] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:11.969912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:11.980821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b374d-ddf4-409d-a81e-86bfa2d66f2d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:11.982268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 778/1548] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:11.984171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e242-9135-424f-a3e9-fd0dd7c99ddf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.984464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e242-9135-424f-a3e9-fd0dd7c99ddf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.985303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e242-9135-424f-a3e9-fd0dd7c99ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.992989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:11.994743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:11.995088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:11.995506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:11.996259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:11.997991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7af9-64dc-4b09-b074-1300759fcfac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.004814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8be3e242-9135-424f-a3e9-fd0dd7c99ddf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.006006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8be3e242-9135-424f-a3e9-fd0dd7c99ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.008305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 771/1549] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:12.009674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d071c4-0bff-489b-9424-ae97100031a0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.009976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d071c4-0bff-489b-9424-ae97100031a0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.010243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d071c4-0bff-489b-9424-ae97100031a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.014391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 779/1550] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.015632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588ef7-7d2d-41f4-9b52-2b6c3aa33e86 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.015999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588ef7-7d2d-41f4-9b52-2b6c3aa33e86 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.016280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5588ef7-7d2d-41f4-9b52-2b6c3aa33e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.030703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-72d071c4-0bff-489b-9424-ae97100031a0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.032500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-72d071c4-0bff-489b-9424-ae97100031a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.034413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b5588ef7-7d2d-41f4-9b52-2b6c3aa33e86 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.035654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5588ef7-7d2d-41f4-9b52-2b6c3aa33e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.040655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 772/1551] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.043698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 780/1552] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.044271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.044807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.044924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844f44d-ea4c-48ce-9485-2f8cb163bfc3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.045176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844f44d-ea4c-48ce-9485-2f8cb163bfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.045461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844f44d-ea4c-48ce-9485-2f8cb163bfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.045624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.066683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.066889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6844f44d-ea4c-48ce-9485-2f8cb163bfc3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.067835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.067968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6844f44d-ea4c-48ce-9485-2f8cb163bfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.079240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 773/1553] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.079415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 781/1554] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.079516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.079617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.079617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.079617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.091508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236da1-918d-44f6-b385-2033f7f0a639 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.096876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.097563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.097848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.098130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.098889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.099199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.099422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.100209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:12.100209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:08:12.100209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:08:12.100817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.101166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ae35d-9c39-411d-8863-fb553ae52ce1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.110223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 782/1555] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.110949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.117900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 774/1556] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.119634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885a044-14ea-438a-b5ab-08b26a8a2418 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.119929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885a044-14ea-438a-b5ab-08b26a8a2418 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.120209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885a044-14ea-438a-b5ab-08b26a8a2418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.129018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51492-6a66-4259-8cfe-a993c923a3d1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.133927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.134841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.135128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.135407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] PATH_INFO: `/v3/projects/9456ea07428140ba96ba166234aeaf7b/users/e800ac0d2ae343f99e89bcdf8c26e9cc/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.138378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5885a044-14ea-438a-b5ab-08b26a8a2418 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.139658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5885a044-14ea-438a-b5ab-08b26a8a2418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.148199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 775/1557] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.149408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.149577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9456ea07428140ba96ba166234aeaf7b, user_id=e800ac0d2ae343f99e89bcdf8c26e9cc, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.149684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.149959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.150733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:12.150733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:08:12.150733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:08:12.153063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaa5e3b-4367-4278-96e2-9efc4e7041a6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.166659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 783/1558] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:12 2022] PUT /identity/v3/projects/9456ea07428140ba96ba166234aeaf7b/users/e800ac0d2ae343f99e89bcdf8c26e9cc/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:12.168059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880e0a-de89-419c-8845-ff2d27589958 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.168307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880e0a-de89-419c-8845-ff2d27589958 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.168588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880e0a-de89-419c-8845-ff2d27589958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.168932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.170153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.187015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-33880e0a-de89-419c-8845-ff2d27589958 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.188289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-33880e0a-de89-419c-8845-ff2d27589958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.236469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 776/1559] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.237392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.246591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1281b-6c85-4968-9a1b-e7300796412c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.250444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 784/1560] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.251815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.252102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.252377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.254707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.255456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.255744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.256021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.256752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.257062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.258995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc965af-0d0d-4243-90bd-4f28647d240b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.259697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:12.271723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 777/1561] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.272847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.273271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.273555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.284404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.285616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.291403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.292583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.293566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 785/1562] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.298150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.303811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b50c-7ff7-4fbd-89de-9e1883000223 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.305549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 778/1563] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.306359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.309977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.310693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.311017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.311295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.311978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.313037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:12.313037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:08:12.313037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:08:12.314220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.316041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0221e5-ab1e-4984-9b3f-6f9edf959d4c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.320684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.321266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.321553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.321834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.322570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.322917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.324520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11e424-38fe-4bd0-b909-5ef5965a8447 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.337526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 779/1564] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.339077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.339751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.340439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.345595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 786/1565] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.345595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.369694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] MFA Rules not processed for user `e800ac0d2ae343f99e89bcdf8c26e9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:12.370956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:12.403781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:12.412998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fda86a-8c12-4ddb-a99f-5a324a860e98 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.419039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.419998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.420425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.420874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.421768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.422204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.424334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.438030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 787/1566] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.438944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.448687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d6255-9a4a-41b5-aa8c-1ea86c0714bb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.450709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 780/1567] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:12.454716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.455365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.455481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.455698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.455976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.456737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.456998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.458588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.465374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec4e8d-dcf3-4e9f-90c2-c8be43339ac9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.470723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.471382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.471507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 788/1568] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.472267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.475102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.475410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.476125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.478726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.482285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6ac52-cf16-4344-af6a-83f67fe1cdfd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.487339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.487975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.488255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.488528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.489392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.489695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.491305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.503202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 789/1569] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.504127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.511076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 781/1570] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.512047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.520064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612278a-6866-4bae-a0c4-784c71cff04f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.573831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226ce80-fb4e-4546-9995-4cdefc21ba01 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.576838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.577496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.577779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.578056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.579026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.579335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:12.579640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.579743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.579947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.580250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.580941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.581275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.582863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.593048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 790/1571] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:12.594694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.604462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5140c3c-db26-4f98-af6d-47d5291c9669 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.610186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.610902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.611191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.611468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.612146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.614204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.616643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 782/1572] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.620531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.635788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 791/1573] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.636579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.646047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06cf8-0b5e-4e29-b9a9-0950c6d8f8a7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.658624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:12.684983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a655e-c61e-4146-accb-2e9310b53a42 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.689758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.690449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.690736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.691076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.691852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.693766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.703818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.704484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.704774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.705055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.705744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.707874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.718066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 783/1574] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.719546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.738268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 792/1575] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.741455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:12.742305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.782116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e5b3-6165-434b-8252-42b12efaa5d4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.803858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ecab9-a9ee-439e-a67a-c75cc3de1bf6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.808848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.809891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.810172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.810449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.811285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.813261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.838314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 793/1576] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.839037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.840233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.840892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.841174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.841740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.842490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.844946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.872153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 784/1577] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.873086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.904258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79c8dc-f7f2-4340-838e-5d856fa35b4f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.932549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc047c-8229-4a6f-b09b-4b06041a030a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:12.936462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:12.937450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.938099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.938494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.938912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.939603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.941625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.962477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:12.963208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:12.963495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:12.963772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:12.964456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:12.966402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:12.967665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 785/1578] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.968466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.986859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 794/1579] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:12.988405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:12.999757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ff3b-1d9b-4c7c-8d07-11c651f0aeb0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.011323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:13.033261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eedf96-0c6d-4da7-833a-3f474565efb7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.037883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.038547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.038860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.039139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.039854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.040874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:13.040874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:08:13.040874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:08:13.042076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.058243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.058961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.059239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.059784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.060514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.062114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.062731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 786/1580] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:12 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.066489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.074853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 795/1581] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.074853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.076079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72cbb9-4a4c-4d03-aa8a-717b9366ffbc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.081311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.081957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.082240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.082516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.082978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31261dd2-c449-40eb-9bad-4820c1056831 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.083263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.085166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.087533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.088176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.088515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.088746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.089418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.091385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.106333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 787/1582] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/users => generated 288 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.107163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.111090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 796/1583] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/users => generated 309 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.111923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.120373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f0d0e-0e11-4381-99bb-1d766d123136 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.121405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71daea2f-578f-434b-af4c-ee80328568ee admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.125550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.126248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.126407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.126571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.126888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.126997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.127191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.127469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.127569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.128293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.129466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.130185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.150644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 788/1584] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.150644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.154317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 797/1585] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/users => generated 292 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.154317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.163160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93892-87f4-4231-be99-9a6adda419c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.165194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0cad-79b3-4f56-b877-cd21fd6b0152 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.169244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.169958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.170082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.170244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.170520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.170660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.170981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.171255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.171352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.171948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.172916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.173848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.183099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 798/1586] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.184011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.198497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 789/1587] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:13.198497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.202622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15ecfb-3f09-4598-8f36-eb810f799f92 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.206221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da791f86-ac45-484c-86a9-a49d2655bea7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.210672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.210672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.210672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.210672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.211085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.211162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.211522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.211729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.211842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.212062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.212815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.214691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.223321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 799/1588] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.224131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.225436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 790/1589] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.226376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.236210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685327-b81e-44d3-a88c-64f08328fe81 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.236752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1346e-d0ea-46bc-9263-7cf7e6e21024 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.240778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.241151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.241409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.241694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.241798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.241974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.242070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.242310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.242681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.243043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.244277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.244601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.254017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 791/1590] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.257823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.257955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 800/1591] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.257955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.267249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f69e2e-5220-4f80-90dc-cf11e677ef48 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.267412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef28b0-7617-4b34-8f49-58ce7b4bab98 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.271789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.272045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.272414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.272700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.272806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.272979 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.273076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.273285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.273690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.274000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.275285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.275610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.286044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 792/1592] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.289518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 801/1593] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.289672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.289779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.296829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbb6e9-8d36-4bf9-8fcc-dbffe1660de6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.298491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20675598-5245-4ab2-8194-72ed6c77498d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.301484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.302121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.302407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.302687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.303171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.303463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.303805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.304088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.304361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.305036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.305143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.306613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.315274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 802/1594] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.316060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.325512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 793/1595] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.325512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.331394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fb96f-cbea-4a9a-a0bf-a359c4f5819a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.335557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4528-570a-4f1f-8176-d588e9224230 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.335968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.336632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.336916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.337190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.337914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.339688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.342750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.343658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.344573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.344857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] PATH_INFO: `/v3/projects/fa10fec302f0464ead56582b81409441/users/ec800c1fb0e74f2caabab848db0432db/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.350619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 803/1596] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.351430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.359581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa10fec302f0464ead56582b81409441, user_id=ec800c1fb0e74f2caabab848db0432db, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.362234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.364706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea31d67-c5cd-451b-a303-71d31f9a97c3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.369959 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.371171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.371629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.371907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] PATH_INFO: `/v3/projects/0f0e26a396324bd686ef730dd8373e83/users/af32479774374fb089f6c6e56f0f9c3d/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.375598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 794/1597] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/fa10fec302f0464ead56582b81409441/users/ec800c1fb0e74f2caabab848db0432db/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.378998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.390104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0e26a396324bd686ef730dd8373e83, user_id=af32479774374fb089f6c6e56f0f9c3d, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.393468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.396572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:13.408520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 804/1598] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/0f0e26a396324bd686ef730dd8373e83/users/af32479774374fb089f6c6e56f0f9c3d/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.409385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.439471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851d50-c5dc-4575-b446-318f05bd093f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.449288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:13.471715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e13307-098d-43aa-a24d-0d96bd8441bf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.476983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.477656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.477943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.478230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.478991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.480686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.490787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 805/1599] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.491663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.499242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.499801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.500632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.501043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.501861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.502933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ac0d0-991b-4e05-a059-d0dae116b61c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.503679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.508071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.508727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.509015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.509293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.510012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.511655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.513427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 795/1600] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.514219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.524412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c3890-01d7-4cdf-807a-c0519ed0ce4f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.525403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 806/1601] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.526191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.532278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.532918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.533201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.533471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.534185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.535779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.536607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ec73-1dd8-4086-b13d-07086efef60c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.541142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.542030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.542313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.542583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] PATH_INFO: `/v3/projects/dc0aa7206a704d45a610dccc24ec8767/users/df44f27783d34370ad83470ffde558a8/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.545450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 796/1602] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.546190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.556157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbbef7e-106e-4ff6-afa1-6fb5aed31f9f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.561151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.561307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0aa7206a704d45a610dccc24ec8767, user_id=df44f27783d34370ad83470ffde558a8, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.562016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.562304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.562743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] PATH_INFO: `/v3/projects/d94c1c234804421487c49cc9df6ff888/users/4549c33736bc413a87460db0c43f5768/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.563972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75a1a7-026c-4bad-b1a5-3e3c47fcf81f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.576776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 807/1603] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/dc0aa7206a704d45a610dccc24ec8767/users/df44f27783d34370ad83470ffde558a8/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.577924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94c1c234804421487c49cc9df6ff888, user_id=4549c33736bc413a87460db0c43f5768, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.578465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.578465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.578641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.580659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.594681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 797/1604] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/d94c1c234804421487c49cc9df6ff888/users/4549c33736bc413a87460db0c43f5768/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.595318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.597088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:13.598288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:13.658322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 808/1605] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:13.659276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.662638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837d91-158e-491f-8abb-0b35cc97f74a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.667334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.668001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.668285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.668471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bf2ab-4cfc-4a9f-9456-75d4e41bdd00 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.668583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.669268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.670879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.673223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.674106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.674508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.674814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] PATH_INFO: `/v3/projects/fb0886382b9e4783a20f03482f10f701/users/82c968fc661c4cd29ecf7c1ce2e3620a/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.684976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 798/1606] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:13.684976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.691035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0886382b9e4783a20f03482f10f701, user_id=82c968fc661c4cd29ecf7c1ce2e3620a, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.693688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.695339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222be0d1-bc5f-42eb-aa3e-47b7365abf39 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.700731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.701750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.702036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.702314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] PATH_INFO: `/v3/projects/ef58452af01448e4b56d2f82a483937b/users/84537e9b76d248d9b1e48d3f29c373b6/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.709834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 809/1607] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/fb0886382b9e4783a20f03482f10f701/users/82c968fc661c4cd29ecf7c1ce2e3620a/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.710618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.722075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef58452af01448e4b56d2f82a483937b, user_id=84537e9b76d248d9b1e48d3f29c373b6, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.724755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.738592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 799/1608] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/ef58452af01448e4b56d2f82a483937b/users/84537e9b76d248d9b1e48d3f29c373b6/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.739458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.754579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:13.784483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37cf58-2e13-4f1e-bf0f-e2a10b95282e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.793400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:13.809574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3883ac9-970f-4435-82ba-6fb73d397528 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.814750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.815658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.815937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.816237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] PATH_INFO: `/v3/projects/c0aec4ac096f4f30b7c357db043925fd/users/0e8bf7bbabc4432fa88002ea5c1dce4c/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.831405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0aec4ac096f4f30b7c357db043925fd, user_id=0e8bf7bbabc4432fa88002ea5c1dce4c, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.834128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b9d85-f061-44e4-9d2b-707407707a57 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.841204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.842097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.842379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.842654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] PATH_INFO: `/v3/projects/d1aca28518cc4f5782c74f34479175f1/users/f9c4512401034796abda50f2fda51887/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.846711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 800/1609] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/c0aec4ac096f4f30b7c357db043925fd/users/0e8bf7bbabc4432fa88002ea5c1dce4c/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.847931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.848225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.848508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.856919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1aca28518cc4f5782c74f34479175f1, user_id=f9c4512401034796abda50f2fda51887, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.860159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020df33-71a0-4cee-ba06-cbcf38ce7904 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.865490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:13.866827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:13.873399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 810/1610] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/d1aca28518cc4f5782c74f34479175f1/users/f9c4512401034796abda50f2fda51887/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.874664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01949-881a-400c-9341-41d445984fd2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.874989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01949-881a-400c-9341-41d445984fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.875822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb01949-881a-400c-9341-41d445984fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.898288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-edb01949-881a-400c-9341-41d445984fd2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:13.899552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-edb01949-881a-400c-9341-41d445984fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:13.935260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 801/1611] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:13.939751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.963239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 811/1612] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:13.964062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb01949-881a-400c-9341-41d445984fd2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:13.974437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb01949-881a-400c-9341-41d445984fd2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.979285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:13.980176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.980460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:13.980735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] PATH_INFO: `/v3/projects/0f0e26a396324bd686ef730dd8373e83/users/af32479774374fb089f6c6e56f0f9c3d/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:13.983193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0e26a396324bd686ef730dd8373e83, user_id=af32479774374fb089f6c6e56f0f9c3d, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:13.986394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6fc2b-8e59-424f-916c-6a23197c1336 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:13.996514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd967c6a-9f30-42a4-a3e0-71bd0289f1a3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:13.998101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 812/1613] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:13 2022] PUT /identity/v3/projects/0f0e26a396324bd686ef730dd8373e83/users/af32479774374fb089f6c6e56f0f9c3d/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:13.999361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1823d0-f0da-4233-8e12-041a199078b6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:13.999790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1823d0-f0da-4233-8e12-041a199078b6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.000983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1823d0-f0da-4233-8e12-041a199078b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.027589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2c1823d0-f0da-4233-8e12-041a199078b6 None None] MFA Rules not processed for user `82c968fc661c4cd29ecf7c1ce2e3620a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.028942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2c1823d0-f0da-4233-8e12-041a199078b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.046439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.047639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.047919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.048191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.070786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:14.088795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:14.096125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 813/1614] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.097483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.097721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.097997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.113293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.114745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb6f4d-b7d5-4ed6-8939-7549f9ac5a16 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.123782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] MFA Rules not processed for user `84537e9b76d248d9b1e48d3f29c373b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.124063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 802/1615] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:14.125716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.126637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe004b-d6ce-499f-9a94-41e1980eb613 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.126951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe004b-d6ce-499f-9a94-41e1980eb613 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.127241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe004b-d6ce-499f-9a94-41e1980eb613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.143936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3dbe004b-d6ce-499f-9a94-41e1980eb613 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.145150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3dbe004b-d6ce-499f-9a94-41e1980eb613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.154793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 803/1616] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.156117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b897c97-cedd-4e91-ae32-88eb58244f33 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.156956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b897c97-cedd-4e91-ae32-88eb58244f33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.157334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b897c97-cedd-4e91-ae32-88eb58244f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.190516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1b897c97-cedd-4e91-ae32-88eb58244f33 None None] MFA Rules not processed for user `f9c4512401034796abda50f2fda51887`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.191805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1b897c97-cedd-4e91-ae32-88eb58244f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.204214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 814/1617] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.208482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.221313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777969b-b862-4173-b1da-0ba86e7c3436 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.225053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.225706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.226196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.226321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.238661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.240044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.251705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 815/1618] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:14.252534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.260505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d707-15c7-4eed-a8c6-eef332d55120 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.269259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.270026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.270387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.270698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.273868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 804/1619] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.275562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.275850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.276125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.304374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.306126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.319914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] MFA Rules not processed for user `af32479774374fb089f6c6e56f0f9c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.321652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.322209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 816/1620] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:14.323137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.335301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f6299-9601-440c-923c-b1c2b06b54dc service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.341141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.341812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.342099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.342376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.343131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.345583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.376991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 817/1621] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:14.378032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.379798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.406144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 805/1622] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.409852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.422647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c519a-9c88-4df3-acce-52ca47545db7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.444881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e6642-1387-4fc9-813a-b304df44c77b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.448869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.449624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.450107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.450479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.471067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:14.489206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.489886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.490171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.490450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.491192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.494123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4048fc6-7da0-4d24-92c6-3dc3bb2be2bb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.495563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:14.528799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 806/1623] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:14.530356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211372a0-1287-432e-b49f-e7b5d5253a18 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.530650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211372a0-1287-432e-b49f-e7b5d5253a18 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.531357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211372a0-1287-432e-b49f-e7b5d5253a18 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.533267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 807/1624] 10.222.0.48 () {54 vars in 961 bytes} [Fri Jul 1 07:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:14.534369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211372a0-1287-432e-b49f-e7b5d5253a18 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.542667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.544202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.555859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 818/1625] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:14.556872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.608252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211372a0-1287-432e-b49f-e7b5d5253a18 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.613738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.614476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.614759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.615070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.615946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.618422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.632818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b5b9c-4073-4444-8a20-fc1ecebcf6b7 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.654294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 808/1626] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:14.658079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.681445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff8c5-b8fc-4738-ad4e-429505508747 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.698560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.699539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.699829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.700107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.700811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.702757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7dba-1513-403f-9153-0072d1ed30af admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.716516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.717151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.717929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.717929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.733068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 809/1627] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/users => generated 295 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:14.735505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.735954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.736407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.772980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:14.778209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:14.821391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.824821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.844713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 819/1628] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:14.845580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.886812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb968b65-a47d-44a5-acad-8538d991c81f service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.900459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.901658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.901658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.901658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.902387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.904567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.927351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 810/1629] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Jul 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:14.928666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.936484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 820/1630] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:14.938065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:14.947359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aff697-def5-4cd3-bca6-e54a2e8125ff None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.955492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852f4be-1e4f-4ac1-a7ea-14e187eabc37 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:14.962645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.963521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:14.963675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.963880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.964772 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:14.964889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.965728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:14.966224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:14.970836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:14.974048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:14.994993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 811/1631] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:14.994993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.009021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae7f7-66fa-4f10-ac21-f77e19e8cd33 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.015339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.017215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.017733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.018165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] PATH_INFO: `/v3/projects/c35f5bd115204e10b4907a4f6e25744e/users/81e8f7b301e74c66a4f7b720126a8f66/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.045121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:15.057822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35f5bd115204e10b4907a4f6e25744e, user_id=81e8f7b301e74c66a4f7b720126a8f66, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.061794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.076135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:15.096526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 812/1632] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:14 2022] PUT /identity/v3/projects/c35f5bd115204e10b4907a4f6e25744e/users/81e8f7b301e74c66a4f7b720126a8f66/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:15.097932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.165875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.179365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.208379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 821/1633] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:15.209310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.232649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef42637-7513-4510-b85a-05ef36c14239 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.305897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6452b5-eff4-4dac-82ec-2217254a7bf1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.316167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.317077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.317664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.317796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] PATH_INFO: `/v3/projects/438bf2aaaa7a47daa4f3dd479766898f/users/8b37d5de39e54fd79988554a0ea241e4/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.329942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.330623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.331357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.331653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.332639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.334048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.350824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 822/1634] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:15.350824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.351118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=438bf2aaaa7a47daa4f3dd479766898f, user_id=8b37d5de39e54fd79988554a0ea241e4, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.352535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.363072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4491184-7a49-4d16-829c-c88d55c5716a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.369613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.370734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.371099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.371377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] PATH_INFO: `/v3/users/e800ac0d2ae343f99e89bcdf8c26e9cc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.372957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e800ac0d2ae343f99e89bcdf8c26e9cc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.374025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:15.374025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 01 07:08:15.374025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: Jul 01 07:08:15.375496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.381030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 813/1635] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:15 2022] PUT /identity/v3/projects/438bf2aaaa7a47daa4f3dd479766898f/users/8b37d5de39e54fd79988554a0ea241e4/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:15.382111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.471467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e800ac0d2ae343f99e89bcdf8c26e9cc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:15.488823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e800ac0d2ae343f99e89bcdf8c26e9cc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:15.489766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e800ac0d2ae343f99e89bcdf8c26e9cc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:15.520630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e8fc6-25d4-4265-8b03-5986e320cc4f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.537874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.539574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.541492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.541814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] PATH_INFO: `/v3/projects/c35f5bd115204e10b4907a4f6e25744e/users/81e8f7b301e74c66a4f7b720126a8f66/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.545244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35f5bd115204e10b4907a4f6e25744e, user_id=81e8f7b301e74c66a4f7b720126a8f66, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.548503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e800ac0d2ae343f99e89bcdf8c26e9cc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:15.549199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.549856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2a998e69-7e9d-4cbe-8892-5648c6576643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e800ac0d2ae343f99e89bcdf8c26e9cc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:15.571572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 814/1636] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:15 2022] PUT /identity/v3/projects/c35f5bd115204e10b4907a4f6e25744e/users/81e8f7b301e74c66a4f7b720126a8f66/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:15.572783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.575203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:15.597058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 823/1637] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:15 2022] DELETE /identity/v3/users/e800ac0d2ae343f99e89bcdf8c26e9cc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:15.598193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd059-174f-46c4-9c9f-73d9746bf424 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.598472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd059-174f-46c4-9c9f-73d9746bf424 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.598826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd059-174f-46c4-9c9f-73d9746bf424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.649339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cc6bd059-174f-46c4-9c9f-73d9746bf424 None None] MFA Rules not processed for user `81e8f7b301e74c66a4f7b720126a8f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:15.650751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc6bd059-174f-46c4-9c9f-73d9746bf424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:15.686757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:15.754452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7b737-08ea-4e14-a915-f8bb27a0fb15 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:15.759983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 824/1638] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:15.761183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30823-65f7-4cfe-a27b-207ff80402bc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.761485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30823-65f7-4cfe-a27b-207ff80402bc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.761771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c30823-65f7-4cfe-a27b-207ff80402bc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.764072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 825/1639] 10.222.0.48 () {54 vars in 912 bytes} [Fri Jul 1 07:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:15.765477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96fe4e-2ca2-4981-8283-b2b130d35e32 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.766343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96fe4e-2ca2-4981-8283-b2b130d35e32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.766631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96fe4e-2ca2-4981-8283-b2b130d35e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.790025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e96fe4e-2ca2-4981-8283-b2b130d35e32 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:15.791726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e96fe4e-2ca2-4981-8283-b2b130d35e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:15.797555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:15.873800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:15.874937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:15.875968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:15.876419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:15.877429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:15.888895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:15.917885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 815/1640] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:15.920687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:15.924363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.039272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 826/1641] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:16.040907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.041310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.041595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.051123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e4f7-8ce4-4869-b9be-6d39a835c42c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.060690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.061539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.061943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.062319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.065820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:16.069121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.085701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:16.087425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:16.107761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 816/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:16.108759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:16.133224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ca9cc-4001-48f7-b4f3-37f8800274e9 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.142934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.143999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.144391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.144773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] PATH_INFO: `/v3/projects/438bf2aaaa7a47daa4f3dd479766898f/users/8b37d5de39e54fd79988554a0ea241e4/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.147747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=438bf2aaaa7a47daa4f3dd479766898f, user_id=8b37d5de39e54fd79988554a0ea241e4, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:16.155758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:16.160862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 827/1643] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:08:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:16.162792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:16.164338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.183209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 817/1644] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:16 2022] PUT /identity/v3/projects/438bf2aaaa7a47daa4f3dd479766898f/users/8b37d5de39e54fd79988554a0ea241e4/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:16.195947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:16.199558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.273618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157c18b-711d-4ee6-9c53-747679add13f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.279343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:16.356874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b4e22-1d47-44ed-b4d5-08789e3fc7ce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.361552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.362229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.362355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.362640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.363104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.363221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.363493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.363818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.461282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:16.461929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:16.464958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020262f8-046a-4359-a721-b37294fa2817 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:16.482017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 828/1645] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:16.483521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.483815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.484100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.490509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:16.492522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:16.532638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 818/1646] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:16.536726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] MFA Rules not processed for user `8b37d5de39e54fd79988554a0ea241e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:16.538253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.701355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 829/1647] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:16.787100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:16.814761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:16.833800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfdc6f-099c-4459-a900-54168f10c1c5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.854064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.857162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.857502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.857741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.861554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:16.926102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539403a-b41d-483b-9e94-1da187e66d6d service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:16.951574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e8f7b301e74c66a4f7b720126a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c35f5bd115204e10b4907a4f6e25744e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:16.960133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:16.961339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:16.962351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:16.965657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:16.966227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:16.969869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:16.979048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:17.014017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 819/1648] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:17.018811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:17.033215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:17.035193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:17.039670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48bf6d-67af-4456-97af-3c2fddd88374 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:17.052790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e8f7b301e74c66a4f7b720126a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c35f5bd115204e10b4907a4f6e25744e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:17.053667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:17.054071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:17.054960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:17.059880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 830/1649] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:17.077165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:17.084571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:17.167129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 820/1650] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:17 2022] POST /identity/v3/projects => generated 354 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:17.171332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:17.194419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96ab1f-4ff7-4717-9ba0-2d72947c1f8b service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:17.309585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e8f7b301e74c66a4f7b720126a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c35f5bd115204e10b4907a4f6e25744e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:17.310398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:17.310747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:17.311165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:17.311971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:17.312350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:17.315785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491996a3-98e7-47fe-9dae-4a6f32d4fd01 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:17.336681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 831/1651] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:17.611872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:17.644161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648446d-3624-462d-bfe3-9c6bbe4e42d1 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:17.666362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81e8f7b301e74c66a4f7b720126a8f66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c35f5bd115204e10b4907a4f6e25744e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:17.667313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:17.667601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:17.668797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] PATH_INFO: `/v3/projects/0dcabec4efe542b8b7c988d986c376f6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:17.672679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0dcabec4efe542b8b7c988d986c376f6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:17.676127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 01 07:08:17.676127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 01 07:08:17.676127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: Jul 01 07:08:17.677472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:17.859096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dcabec4efe542b8b7c988d986c376f6', 'request_id': 'req-2336d136-3801-4121-8f4c-93b0dcc60006'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:17.878534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dcabec4efe542b8b7c988d986c376f6', 'request_id': 'req-2336d136-3801-4121-8f4c-93b0dcc60006'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:17.904372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dcabec4efe542b8b7c988d986c376f6', 'request_id': 'req-2336d136-3801-4121-8f4c-93b0dcc60006'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:17.926561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] The token cache is being invalidate because project 0dcabec4efe542b8b7c988d986c376f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:17.926882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:17.927581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2336d136-3801-4121-8f4c-93b0dcc60006 tempest-VolumeQuotaClassesTest-1561011293 tempest-VolumeQuotaClassesTest-1561011293-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:17.938964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 821/1652] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:17 2022] DELETE /identity/v3/projects/0dcabec4efe542b8b7c988d986c376f6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:18.111279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a58-9d60-4c59-8b1d-4a425c17d223 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.111893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a58-9d60-4c59-8b1d-4a425c17d223 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.111893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7a58-9d60-4c59-8b1d-4a425c17d223 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.113559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 832/1653] 10.222.0.48 () {54 vars in 961 bytes} [Fri Jul 1 07:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:18.121579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9a0fe-2770-4448-ab21-01b6ebd5794e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.121877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9a0fe-2770-4448-ab21-01b6ebd5794e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.122166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9a0fe-2770-4448-ab21-01b6ebd5794e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.168028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10c9a0fe-2770-4448-ab21-01b6ebd5794e None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:18.170480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10c9a0fe-2770-4448-ab21-01b6ebd5794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.264817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:18.341879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 822/1654] 10.222.0.48 () {58 vars in 1058 bytes} [Fri Jul 1 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:18.348573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.349884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.350189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.352427 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 833/1655] 10.222.0.48 () {54 vars in 912 bytes} [Fri Jul 1 07:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:18.357905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.358217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.358476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.400500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:18.402294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.403541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:18.405105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.423443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 823/1656] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:18.433804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:18.436250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.534842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfed15-b612-403a-bf27-bceed16ee312 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:18.541604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:18.542743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.543242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.543681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] PATH_INFO: `/v3/users/82c968fc661c4cd29ecf7c1ce2e3620a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.545272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c968fc661c4cd29ecf7c1ce2e3620a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:18.548356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:18.555367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b8f1-1aa7-4f00-af31-44ffe71c89de None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:18.561585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:18.562022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.562446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.563231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.583664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:18.585548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:18.604601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 824/1657] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:18.656006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c968fc661c4cd29ecf7c1ce2e3620a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:18.668421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c968fc661c4cd29ecf7c1ce2e3620a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:18.669597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c968fc661c4cd29ecf7c1ce2e3620a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:18.704255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:18.705461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c968fc661c4cd29ecf7c1ce2e3620a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:18.705935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.706310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f340ad18-9e1f-4c63-bf4a-93391c7abf51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c968fc661c4cd29ecf7c1ce2e3620a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:18.718284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:18.726178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 834/1658] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:18 2022] DELETE /identity/v3/users/82c968fc661c4cd29ecf7c1ce2e3620a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:18.728006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.728422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.728810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.736331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d920f29-0809-438e-b24f-061a576f5842 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:18.759440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:18.760845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.861616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:18.863149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:18.863653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:18.864008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] PATH_INFO: `/v3/projects/9456ea07428140ba96ba166234aeaf7b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:18.866951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9456ea07428140ba96ba166234aeaf7b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:18.872617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:18.883980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 835/1659] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:18.898399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:18.900197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:18.976852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9456ea07428140ba96ba166234aeaf7b', 'request_id': 'req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.059257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9456ea07428140ba96ba166234aeaf7b', 'request_id': 'req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.059257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9456ea07428140ba96ba166234aeaf7b', 'request_id': 'req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.059257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] The token cache is being invalidate because project 9456ea07428140ba96ba166234aeaf7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:19.059257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.062499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.064109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b944c-3bd1-44e7-95e9-9fdbcfef791a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:19.072708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 825/1660] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:18 2022] DELETE /identity/v3/projects/9456ea07428140ba96ba166234aeaf7b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:19.082116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:19.084399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:19.091430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:19.224889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:19.226271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:19.226873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:19.227358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:19.238754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:19.271471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08204f9e-8f8f-45d5-a3a5-ec0a7adfed0d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:19.284834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:19.287275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:19.289273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:19.289810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] PATH_INFO: `/v3/users/af32479774374fb089f6c6e56f0f9c3d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:19.293758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af32479774374fb089f6c6e56f0f9c3d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:19.296648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:19.360480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:19.364144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:19.371830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:19.382756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 836/1661] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:19.405604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af32479774374fb089f6c6e56f0f9c3d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.419733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af32479774374fb089f6c6e56f0f9c3d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.420859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af32479774374fb089f6c6e56f0f9c3d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.443699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af32479774374fb089f6c6e56f0f9c3d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.445498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af32479774374fb089f6c6e56f0f9c3d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:19.477638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 826/1662] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:19 2022] DELETE /identity/v3/users/af32479774374fb089f6c6e56f0f9c3d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:20.914273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:20.916270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:20.934721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8144-1da3-4d1e-8705-5f8c2a1a7a58 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:21.079591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:21.080717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:21.081005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:21.081280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] PATH_INFO: `/v3/projects/fb0886382b9e4783a20f03482f10f701` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:21.087019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb0886382b9e4783a20f03482f10f701)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:21.088970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:21.185425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb0886382b9e4783a20f03482f10f701', 'request_id': 'req-ec1a9ff6-625b-4767-91c1-5c3451659159'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.200968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb0886382b9e4783a20f03482f10f701', 'request_id': 'req-ec1a9ff6-625b-4767-91c1-5c3451659159'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.228988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb0886382b9e4783a20f03482f10f701', 'request_id': 'req-ec1a9ff6-625b-4767-91c1-5c3451659159'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.253442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] The token cache is being invalidate because project fb0886382b9e4783a20f03482f10f701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:21.254126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.255013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.257590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 837/1663] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:20 2022] DELETE /identity/v3/projects/fb0886382b9e4783a20f03482f10f701 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:21.261469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:21.263518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:21.327547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f31913-fece-4575-98a0-3af9e0cb6e06 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:21.335419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:21.341410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:21.472406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:21.477809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:21.479515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:21.480665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] PATH_INFO: `/v3/users/81e8f7b301e74c66a4f7b720126a8f66` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:21.483674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e8f7b301e74c66a4f7b720126a8f66)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:21.488286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:21.513176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a9ff6-625b-4767-91c1-5c3451659159 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:21.525943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:21.528041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:21.528523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:21.529515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] PATH_INFO: `/v3/projects/0f0e26a396324bd686ef730dd8373e83` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:21.532138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0e26a396324bd686ef730dd8373e83)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:21.534680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:21.547154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:21.602091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e8f7b301e74c66a4f7b720126a8f66'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.623298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e8f7b301e74c66a4f7b720126a8f66'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.623996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e8f7b301e74c66a4f7b720126a8f66'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.659368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e8f7b301e74c66a4f7b720126a8f66'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.660237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d1945e74-b7cd-44a5-bb63-0f93b6b11326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e8f7b301e74c66a4f7b720126a8f66'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.661056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:21.713834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0e26a396324bd686ef730dd8373e83', 'request_id': 'req-8cd343a0-2e96-4a37-aebc-c8e778c049eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.721922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 827/1664] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:21 2022] DELETE /identity/v3/users/81e8f7b301e74c66a4f7b720126a8f66 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:21.729990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0e26a396324bd686ef730dd8373e83', 'request_id': 'req-8cd343a0-2e96-4a37-aebc-c8e778c049eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.752030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0e26a396324bd686ef730dd8373e83', 'request_id': 'req-8cd343a0-2e96-4a37-aebc-c8e778c049eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.774670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] The token cache is being invalidate because project 0f0e26a396324bd686ef730dd8373e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:21.775013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.775678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8cd343a0-2e96-4a37-aebc-c8e778c049eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:21.781493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 838/1665] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:21 2022] DELETE /identity/v3/projects/0f0e26a396324bd686ef730dd8373e83 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:21.785336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:21.785553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:21.785816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:21.839129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:21.841267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:21.995416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 828/1666] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:22.004302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43027e73-662d-4993-a155-3fb30594855d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.004814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43027e73-662d-4993-a155-3fb30594855d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.004903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43027e73-662d-4993-a155-3fb30594855d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.051976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-43027e73-662d-4993-a155-3fb30594855d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:22.057994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-43027e73-662d-4993-a155-3fb30594855d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:22.085075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.085602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 839/1667] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:22.098704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6a63-15b7-4cfd-8f8b-b5e964b90651 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.113714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.114411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.114698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.116832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.117598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:22.122118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.125943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.156001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 829/1668] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:22.158337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43027e73-662d-4993-a155-3fb30594855d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.170736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43027e73-662d-4993-a155-3fb30594855d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.183808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.185044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.185479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.185899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.186752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.189610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.258098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 840/1669] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:22 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:22.262549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.281123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858df8-65f1-46b0-b93b-42e961fb456b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.457223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.458002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.458300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.458637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.459494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.465236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.540226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 830/1670] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:22 2022] POST /identity/v3/users => generated 294 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:22.544518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.562136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345dcfa-da06-4123-898b-8fe53e293cba admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.578686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.579624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.579920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.580198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.581411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.588048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.607481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 841/1671] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:22.612444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.625326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7180e9-39d6-4d92-949a-37f1e88940e4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.632324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.633114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.633475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.633826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.634645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.639220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f5542-7e59-4a35-a07e-31bf38db1e5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.659121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 831/1672] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:22.663369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.679918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3630-b8ce-409c-86fa-8f2e903e29f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:22.687440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:22.691485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.692277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.692277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] PATH_INFO: `/v3/projects/ab44d9d0e47046f1b834f2a410e73cbd/users/8d9a1a137493440e9416d7f7d99783f1/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.729979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab44d9d0e47046f1b834f2a410e73cbd, user_id=8d9a1a137493440e9416d7f7d99783f1, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:22.733020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:22.773282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 842/1673] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:22 2022] PUT /identity/v3/projects/ab44d9d0e47046f1b834f2a410e73cbd/users/8d9a1a137493440e9416d7f7d99783f1/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:22.776294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:22.776824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:22.777287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:22.791851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:22.826797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] MFA Rules not processed for user `8d9a1a137493440e9416d7f7d99783f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:22.829256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:22.956193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:22.958243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 832/1674] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:23.112356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43f09c-30ea-4ab2-9d8d-db90275263c4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.127288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.128589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.129012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.129409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.130398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.132847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.195626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 843/1675] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:22 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:23.199713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.212008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544dcde-92bc-439e-b9d0-f8487e80c654 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.314986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.316712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.317342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.317945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.319061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.321639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.373261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 833/1676] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:23 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:23.384074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.400075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e064a0f-45af-4bea-9c18-9f45148e497c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.406189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.407027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.407382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.407729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.408636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.410643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.423846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 844/1677] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:23.427099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.440616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a815a3-5140-43b5-a45d-01c3c41d0a5c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.448099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.448943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.449228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.449876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.450854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.452844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4a697-0fa8-4d7e-8267-41f872e87116 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.470969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 834/1678] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:23.473931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.489262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90044137-b584-4a01-8737-1a1fc40309b1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.501020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.503856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.507794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.508242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] PATH_INFO: `/v3/projects/cc982fa59ee2420fb64af5c29aef9469/users/d613b95312ea474f90daa63aa74fad2a/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.560888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc982fa59ee2420fb64af5c29aef9469, user_id=d613b95312ea474f90daa63aa74fad2a, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.565172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.604695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 845/1679] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:23 2022] PUT /identity/v3/projects/cc982fa59ee2420fb64af5c29aef9469/users/d613b95312ea474f90daa63aa74fad2a/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:08:23.616886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.617542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.618031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.653901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] MFA Rules not processed for user `d613b95312ea474f90daa63aa74fad2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:23.655541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:23.711952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.715507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:23.739204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:23.803645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 835/1680] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:23.808415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:23.817993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:23.873510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbb779-2d36-471d-8a6c-d730052c72fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.880843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.882071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.882492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.883339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] PATH_INFO: `/v3/projects/c35f5bd115204e10b4907a4f6e25744e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.886403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35f5bd115204e10b4907a4f6e25744e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.904008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:23.958878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6722da-bfe5-48e3-9a64-f742286fd4d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:23.971694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:23.972594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:23.972706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:23.973179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:23.973625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:23.978281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.023009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35f5bd115204e10b4907a4f6e25744e', 'request_id': 'req-fce55f1b-485a-4fe3-918d-561de382e90d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:24.037329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35f5bd115204e10b4907a4f6e25744e', 'request_id': 'req-fce55f1b-485a-4fe3-918d-561de382e90d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:24.046458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 836/1681] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:23 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:24.047705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.058387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35f5bd115204e10b4907a4f6e25744e', 'request_id': 'req-fce55f1b-485a-4fe3-918d-561de382e90d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:24.091762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] The token cache is being invalidate because project c35f5bd115204e10b4907a4f6e25744e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:24.094014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:24.098311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fce55f1b-485a-4fe3-918d-561de382e90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:24.104259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 846/1682] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:23 2022] DELETE /identity/v3/projects/c35f5bd115204e10b4907a4f6e25744e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:24.110472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763fc3-92b2-40ba-a741-625aa5d9cd96 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.111758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763fc3-92b2-40ba-a741-625aa5d9cd96 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.113629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd763fc3-92b2-40ba-a741-625aa5d9cd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.147709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bd763fc3-92b2-40ba-a741-625aa5d9cd96 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:24.149467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bd763fc3-92b2-40ba-a741-625aa5d9cd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:24.225323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2087a32-66b5-481e-86bb-ba336cddc82b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.234224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.235459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.235743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.236020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.236731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.238682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.283377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 837/1683] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:24.287316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.290246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:24.310433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c414fd1-f6a2-4c0e-abfc-5d9a264092d6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.316239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.316921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.317207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.317484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.318254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.319972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.326474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 847/1684] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:24.328787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.331517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.331931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.332314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 838/1685] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:24.336360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.349543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921b035-5241-4a69-9333-86b4f8ce864a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.355373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.356726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.357071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.357419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] PATH_INFO: `/v3/projects/7d72f2d1dc094c2fa003f800789ea450/users/fb709cfd4e9c46759fcb9723717b5c91/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.377166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d72f2d1dc094c2fa003f800789ea450, user_id=fb709cfd4e9c46759fcb9723717b5c91, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.380119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.385570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:24.388972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:24.403110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 839/1686] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:24 2022] PUT /identity/v3/projects/7d72f2d1dc094c2fa003f800789ea450/users/fb709cfd4e9c46759fcb9723717b5c91/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:24.405854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.470328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:24.482462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:24.494638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb4cc6-8858-4e35-ab67-744b72f47a3b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.503060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.503888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.504318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.504696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.505953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.510500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.542351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 840/1687] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:24.545721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.558456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 848/1688] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:24.560275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.574287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ba7f9-2a45-4f86-883f-0fb3010e4b36 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.582809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2581d-443a-446c-a244-bca3513e8880 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.590486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.591703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.592409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.592409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] PATH_INFO: `/v3/projects/7d72f2d1dc094c2fa003f800789ea450/users/fb709cfd4e9c46759fcb9723717b5c91/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.600520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d72f2d1dc094c2fa003f800789ea450, user_id=fb709cfd4e9c46759fcb9723717b5c91, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.600814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.601503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.601786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.602063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.602827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:24.603125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.604712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f381f8-e76b-40ea-8631-e87b0dc1da4f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.604836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.637135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 841/1689] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:24 2022] PUT /identity/v3/projects/7d72f2d1dc094c2fa003f800789ea450/users/fb709cfd4e9c46759fcb9723717b5c91/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:24.646450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.651401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 849/1690] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:24.653226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.653517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.653990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.706253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] MFA Rules not processed for user `fb709cfd4e9c46759fcb9723717b5c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:24.707613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:24.793010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5dd8f-cdb8-4bfd-8bcd-58c41f2069a8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:24.801019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:24.801708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:24.801998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:24.802341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:24.803565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:24.805387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:24.842291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 842/1691] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/projects => generated 365 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:24.847368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.851624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 850/1692] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:24.862702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:24.864195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:24.990218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8a069-e3bd-49b1-904b-3516a9be8cdd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:25.007439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9f083-d6c0-49dd-a545-1c1b811f46aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:25.012475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:25.013991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.014564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.014906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.017034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:25.023024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:25.078843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 843/1693] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:24 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:25.080109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:25.099974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe8376-40a5-422f-a529-eb921a66f8bb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:25.105138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:25.107088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:25.107546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.107820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.107933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.108110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.108386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.109673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.110556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:25.112722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:25.135999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 844/1694] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:25.142537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:25.163076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a3e7d-e166-401d-8b73-f9b5aa3ecc0f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:25.169594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:25.172364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:25.173396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.173837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.174225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.177572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:25.180243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:25.215644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 845/1695] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:08:25.219507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:25.239447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f8ad4-3a3f-4139-af8b-98cce4d545f9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:25.270389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:25.271484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.271864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.272244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] PATH_INFO: `/v3/projects/637264ea67d14e9da3d76d70e761d3b1/users/9817dd9a74034e1cac8149dc2340c0d1/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.293977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:25.294966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] RBAC: Authorizing `identity:create_grant(project_id=637264ea67d14e9da3d76d70e761d3b1, user_id=9817dd9a74034e1cac8149dc2340c0d1, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:25.298434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:25.324450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d63368-099c-459b-9330-ae819abe309e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:25.332909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:25.360117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 846/1696] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:25 2022] PUT /identity/v3/projects/637264ea67d14e9da3d76d70e761d3b1/users/9817dd9a74034e1cac8149dc2340c0d1/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:08:25.369055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 851/1697] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:25.372660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:25.372660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:25.372901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:25.450481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] MFA Rules not processed for user `9817dd9a74034e1cac8149dc2340c0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:25.452256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:26.012038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.024948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 847/1698] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 653 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:26.031022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b67fa-cae4-4c9b-8296-ce2a6954fcdc service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:26.220013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:26.220704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:26.220989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:26.221271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:26.221972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:26.223956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:26.258595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 852/1699] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:26 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:26.264490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.283383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66397c-f367-4bbf-99cc-9ff80f9aeb3b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:26.482850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:26.487432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:26.487833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:26.488129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:26.489980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:26.492364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:26.549159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 848/1700] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:26 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:26.550617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.585666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b28ef0-b273-463a-a34f-89ca1c81cdc0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:26.606930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:26.614632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:26.614997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:26.615286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:26.616939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:26.623410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:26.659997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 853/1701] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:26.664630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.705929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974da8bd-aaf4-42cd-8121-a8457e2879ed admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:26.723237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:26.724264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:26.724705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:26.724977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] PATH_INFO: `/v3/projects/a9b3a6a6383f41e59da269deb4c1c460/users/596d5d12c7a84cbb807303979fcdce34/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:26.763958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b3a6a6383f41e59da269deb4c1c460, user_id=596d5d12c7a84cbb807303979fcdce34, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:26.766809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:26.790872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 849/1702] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:26 2022] PUT /identity/v3/projects/a9b3a6a6383f41e59da269deb4c1c460/users/596d5d12c7a84cbb807303979fcdce34/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:26.793802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.809650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c145c8-c40e-4825-8d31-ed6eca2b023b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:26.944042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:26.946743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:27.007476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:27.008260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.008447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.008724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.009511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:27.011152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:27.036023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 854/1703] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:08:27.036980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:27.058712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63713a60-d52c-406b-b6b1-58c4d2e195c0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:27.070663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:27.071699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.071935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.072221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] PATH_INFO: `/v3/projects/a9b3a6a6383f41e59da269deb4c1c460/users/596d5d12c7a84cbb807303979fcdce34/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.081164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b3a6a6383f41e59da269deb4c1c460, user_id=596d5d12c7a84cbb807303979fcdce34, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:27.084144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e83100-c9dc-4ada-a768-c0162b0218aa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:27.115716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 855/1704] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:27 2022] PUT /identity/v3/projects/a9b3a6a6383f41e59da269deb4c1c460/users/596d5d12c7a84cbb807303979fcdce34/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:27.125861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fdf43-ad69-411a-8016-749194e76463 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.126152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fdf43-ad69-411a-8016-749194e76463 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.126686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fdf43-ad69-411a-8016-749194e76463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.155692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba40941-ed5c-436a-91f8-c704cc70c7c1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:27.171468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:27.181217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:27.193200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-819fdf43-ad69-411a-8016-749194e76463 None None] MFA Rules not processed for user `596d5d12c7a84cbb807303979fcdce34`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:27.221881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-819fdf43-ad69-411a-8016-749194e76463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:27.299004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:27.300130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.300411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.302967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.307112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:27.478120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 856/1705] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:27.496169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fdf43-ad69-411a-8016-749194e76463 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:27.511870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fdf43-ad69-411a-8016-749194e76463 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:27.523373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:27.524278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.524278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.524724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.536116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:27.539000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:27.557330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 850/1706] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:27.557574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:27.559071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e279c23-74a5-4ce7-b5be-5d45c063774e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:27.599271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 857/1707] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:27.799680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:27.815497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d7a54-d79c-485e-bda7-b769ed6865de service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:27.826218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:27.826929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.827757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.828890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.882731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd5c6-f1dd-4148-8058-8db966f0ba5d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:27.883043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd5c6-f1dd-4148-8058-8db966f0ba5d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:27.883319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd5c6-f1dd-4148-8058-8db966f0ba5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:27.984621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bc9dd5c6-f1dd-4148-8058-8db966f0ba5d None None] MFA Rules not processed for user `ec800c1fb0e74f2caabab848db0432db`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:27.988236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc9dd5c6-f1dd-4148-8058-8db966f0ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:28.048258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:28.069379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:28.109842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 851/1708] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:28.155183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:28.173843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 858/1709] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:28.175787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd3b4-d18a-47ae-9ae1-6cec8543e4b5 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:28.186209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:28.186503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:28.186976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:28.193134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:28.196997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:28.198155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:28.201486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:28.215124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:28.244329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] MFA Rules not processed for user `ec800c1fb0e74f2caabab848db0432db`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:28.248645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:28.281830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 859/1710] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:28.293206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:28.294886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:28.430620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:28.437891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:28.439653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff7183-1f6c-4aee-8559-79fff2cd6f0f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:28.500568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 852/1711] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:28.510662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70331236-63d1-4ba6-aa6a-a74e7689c471 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:28.526444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:28.527397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:28.527484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:28.527802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:28.546087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:28.555688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:28.558191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19659460-62e2-433d-938a-b8d4db3ae2cb service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:28.595195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 860/1712] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:29.620345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:29.620896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:29.620896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:29.622666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 853/1713] 10.222.0.48 () {54 vars in 912 bytes} [Fri Jul 1 07:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:29.635315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c907-11d9-4466-9278-5cec71a1ecfa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:29.635770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c907-11d9-4466-9278-5cec71a1ecfa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:29.635850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4c907-11d9-4466-9278-5cec71a1ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:29.696023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74c4c907-11d9-4466-9278-5cec71a1ecfa None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:29.697371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74c4c907-11d9-4466-9278-5cec71a1ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:29.848172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:29.848606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 861/1714] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:08:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:29.850897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:29.906667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bba9d0-2b0c-484e-8d70-d87fcc7f2172 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:29.930753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:29.957208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:29.957656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:29.958082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:29.969266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:29.971120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031fec7-0736-4d9c-832f-2f0d9c127650 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:29.993981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 854/1715] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:30.992522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee2a1-365b-43d7-a44d-e33eae26385e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:30.993048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee2a1-365b-43d7-a44d-e33eae26385e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:30.993144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ee2a1-365b-43d7-a44d-e33eae26385e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:30.994573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 862/1716] 10.222.0.48 () {54 vars in 960 bytes} [Fri Jul 1 07:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:31.008255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007787b-248a-4d97-bda2-c5b280634e5f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.009517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007787b-248a-4d97-bda2-c5b280634e5f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.011258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007787b-248a-4d97-bda2-c5b280634e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.061022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a007787b-248a-4d97-bda2-c5b280634e5f None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:31.067151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a007787b-248a-4d97-bda2-c5b280634e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.200337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 855/1717] 10.222.0.48 () {58 vars in 1057 bytes} [Fri Jul 1 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:31.210728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ee2a1-365b-43d7-a44d-e33eae26385e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:31.230955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ee2a1-365b-43d7-a44d-e33eae26385e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:31.245834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:31.246596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.247996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.248460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.252955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.268482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:31.271146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c59ce-ad64-4dbb-91a6-a3f98cd03561 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:31.286304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 863/1718] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:31.483110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e376ff-74c3-43ab-a57b-70b94dad8b12 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.483781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e376ff-74c3-43ab-a57b-70b94dad8b12 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.484328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e376ff-74c3-43ab-a57b-70b94dad8b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.544826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.545123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.545406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.598173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e1e376ff-74c3-43ab-a57b-70b94dad8b12 None None] MFA Rules not processed for user `0e8bf7bbabc4432fa88002ea5c1dce4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:31.599720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e1e376ff-74c3-43ab-a57b-70b94dad8b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.613006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] MFA Rules not processed for user `df44f27783d34370ad83470ffde558a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:31.614236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.696813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 856/1719] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:31.719961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af7e5-7f1b-4846-af35-862bc534f8da None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.720328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af7e5-7f1b-4846-af35-862bc534f8da None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.720680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24af7e5-7f1b-4846-af35-862bc534f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.743425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 864/1720] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:31.744786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:31.770841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a24af7e5-7f1b-4846-af35-862bc534f8da None None] MFA Rules not processed for user `4549c33736bc413a87460db0c43f5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:31.772461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a24af7e5-7f1b-4846-af35-862bc534f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.790819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443ba55-bee2-4c68-bb98-49d8c7118b83 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:31.805996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:31.806968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.811525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.812117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.832490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:31.836530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:31.871286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 865/1721] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:31.872338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:31.876073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:31.917099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:31.931651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a7097-8475-4ab6-9fab-8671846d722c service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:31.946903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 857/1722] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:31.950275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:31.951858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.952152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.952433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.955646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9fbb6-9231-441c-8c2d-9d23c0953d05 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:31.956173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9fbb6-9231-441c-8c2d-9d23c0953d05 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:31.957206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9fbb6-9231-441c-8c2d-9d23c0953d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:31.974180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:31.978597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:31.985620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-86e9fbb6-9231-441c-8c2d-9d23c0953d05 None None] MFA Rules not processed for user `4549c33736bc413a87460db0c43f5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:31.987876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-86e9fbb6-9231-441c-8c2d-9d23c0953d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:32.004584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 866/1723] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:32.010102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 858/1724] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:32.165039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:32.183020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9f082-9157-41af-b995-04b4c725d4e1 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:32.195445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:32.196169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:32.196461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:32.196890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:32.227765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:32.229568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf89d06-45ed-4041-9fd1-16d30054549c service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:32.246750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 867/1725] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:34.084763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8b47-dd59-427f-bb75-a7c9d41cb903 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:34.085767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8b47-dd59-427f-bb75-a7c9d41cb903 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:34.086313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8b47-dd59-427f-bb75-a7c9d41cb903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:34.123794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-224c8b47-dd59-427f-bb75-a7c9d41cb903 None None] MFA Rules not processed for user `4549c33736bc413a87460db0c43f5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:34.134663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-224c8b47-dd59-427f-bb75-a7c9d41cb903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:34.153313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 859/1726] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:34.972345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111f5-af04-48ec-90ce-cc80d302cf41 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:34.973344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111f5-af04-48ec-90ce-cc80d302cf41 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:34.973771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111f5-af04-48ec-90ce-cc80d302cf41 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:34.975560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 868/1727] 10.222.0.48 () {54 vars in 912 bytes} [Fri Jul 1 07:08:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 01 07:08:34.988629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558282-2483-4cb5-a355-a97907d69af7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:34.988920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558282-2483-4cb5-a355-a97907d69af7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:34.989210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d558282-2483-4cb5-a355-a97907d69af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:35.030373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d558282-2483-4cb5-a355-a97907d69af7 None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:35.032226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d558282-2483-4cb5-a355-a97907d69af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:35.048238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 860/1728] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:35.094112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18111f5-af04-48ec-90ce-cc80d302cf41 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:35.113191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18111f5-af04-48ec-90ce-cc80d302cf41 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:35.120545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:35.121444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:35.121870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:35.122297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:35.145930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:35.147989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1591ff-255e-413d-90a8-653b3c6286c2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:35.163612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 869/1729] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:35.738574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db44570-cd8a-4c14-8328-fb327d90555d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:35.739315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db44570-cd8a-4c14-8328-fb327d90555d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:35.739315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db44570-cd8a-4c14-8328-fb327d90555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:35.776782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5db44570-cd8a-4c14-8328-fb327d90555d None None] MFA Rules not processed for user `df44f27783d34370ad83470ffde558a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:35.781956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5db44570-cd8a-4c14-8328-fb327d90555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:35.803637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 861/1730] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:37.155665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:37.157251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:37.158049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:37.220913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] MFA Rules not processed for user `0e8bf7bbabc4432fa88002ea5c1dce4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:37.222410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:37.271659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 870/1731] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:37.639242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3d472-7350-4861-9e91-12d935046bb4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:37.640346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3d472-7350-4861-9e91-12d935046bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:37.640634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3d472-7350-4861-9e91-12d935046bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:37.666878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1cc3d472-7350-4861-9e91-12d935046bb4 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:37.668429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1cc3d472-7350-4861-9e91-12d935046bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:37.801032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 862/1732] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:08:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:38.076399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:38.078627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:38.093770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:38.355370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a578-7f73-4a02-ae16-3d0e5ce35039 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:38.362800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:38.363750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:38.364195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:38.364697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:38.380669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:38.382690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:38.399595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 871/1733] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:08:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:08:39.454008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c092f8-dabb-4666-a8b1-ab13ab15c0da None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:39.454914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c092f8-dabb-4666-a8b1-ab13ab15c0da None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:39.455427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c092f8-dabb-4666-a8b1-ab13ab15c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:39.490139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-29c092f8-dabb-4666-a8b1-ab13ab15c0da None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:39.493283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-29c092f8-dabb-4666-a8b1-ab13ab15c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:39.504933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 863/1734] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Jul 1 07:08:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:39.603706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:39.605314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:39.628379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb87d7-4b26-4028-af43-198a17348645 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:39.634500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:39.635548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:39.636039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:39.636540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:39.657463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:39.659257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f1adb-8adb-4544-b0c0-03b3b0c2b43d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:39.676209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 872/1735] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:41.971776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5918d-d18e-4d7b-ad2e-8c5ef284fbb1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:41.972714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5918d-d18e-4d7b-ad2e-8c5ef284fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:41.973148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5918d-d18e-4d7b-ad2e-8c5ef284fbb1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:41.975665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 864/1736] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:41.985664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:41.986597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:41.987086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:42.043068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:42.047367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:42.066181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5918d-d18e-4d7b-ad2e-8c5ef284fbb1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:42.083738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5918d-d18e-4d7b-ad2e-8c5ef284fbb1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:42.101459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:42.103326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:42.103768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:42.105266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:42.108039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 873/1737] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:08:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:42.110695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:42.275972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:42.277816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:42.298216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 865/1738] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:42.497741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:42.517387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b806fb2-4b2c-419b-974e-bf3309d78482 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:42.535479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:42.537357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:42.538341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:42.538854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:42.559903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:42.563532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f865b-f32d-4a47-9e5f-117d3097348e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:42.578927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 874/1739] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:46.854418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:46.981996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919db6d-62e2-4fc1-9294-88aaea45b5b2 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:46.990190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:46.991414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:46.991870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:46.992148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] PATH_INFO: `/v3/users/9817dd9a74034e1cac8149dc2340c0d1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:46.993598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9817dd9a74034e1cac8149dc2340c0d1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:46.995652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:47.012787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:47.107526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9817dd9a74034e1cac8149dc2340c0d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.118562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9817dd9a74034e1cac8149dc2340c0d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.123182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9817dd9a74034e1cac8149dc2340c0d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.140559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9817dd9a74034e1cac8149dc2340c0d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.141489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9817dd9a74034e1cac8149dc2340c0d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.159344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 866/1740] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:46 2022] DELETE /identity/v3/users/9817dd9a74034e1cac8149dc2340c0d1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:47.164049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:47.164623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:47.164623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:47.183885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:47.185442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:47.274816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 875/1741] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:47.284012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:47.286963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:47.396879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ee481-9ca8-4820-a301-f5724a948851 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:47.408029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:47.408706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:47.408994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:47.409272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:47.426577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:47.428388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3116ec-84d1-4888-b3ee-fa4a2b2a571e service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:47.444565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 867/1742] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:47.474052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:47.475508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:47.488183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b505bf1-0f2e-4f59-98cc-32580de9e548 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:47.492845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:47.493744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:47.495119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:47.495617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] PATH_INFO: `/v3/users/8d9a1a137493440e9416d7f7d99783f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:47.497027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9a1a137493440e9416d7f7d99783f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:47.499003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:47.557326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9a1a137493440e9416d7f7d99783f1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.567455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9a1a137493440e9416d7f7d99783f1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.568245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9a1a137493440e9416d7f7d99783f1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.574692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:47.601752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9a1a137493440e9416d7f7d99783f1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.602497 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9a1a137493440e9416d7f7d99783f1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:47.622538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 876/1743] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:47 2022] DELETE /identity/v3/users/8d9a1a137493440e9416d7f7d99783f1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:47.627199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:47.627487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:47.627803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:47.661812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:47.663122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:47.778576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 868/1744] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:47.786599 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:47.788176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:47.915591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eaa34-788b-491d-8268-5f8f7fbd8b88 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:47.923572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:47.924250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:47.925411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:47.925712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:47.939237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:47.940868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:47.955734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 877/1745] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:47.985819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:47.988930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.021615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cbd8a-0cc9-424d-89d6-6f08bed17ce8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.028797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:48.029971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.030397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.030925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] PATH_INFO: `/v3/users/f9c4512401034796abda50f2fda51887` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.031615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:48.032898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c4512401034796abda50f2fda51887)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:48.033255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.034929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:48.058028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521840-72fb-4220-8697-2224178b32bc service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.063449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:48.065140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.065751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.066085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] PATH_INFO: `/v3/projects/637264ea67d14e9da3d76d70e761d3b1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.067900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=637264ea67d14e9da3d76d70e761d3b1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:48.070157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:48.081661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:48.145730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637264ea67d14e9da3d76d70e761d3b1', 'request_id': 'req-f0274369-c145-47d6-911f-39cf0c5d21e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.166509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c4512401034796abda50f2fda51887'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.178391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c4512401034796abda50f2fda51887'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.179669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c4512401034796abda50f2fda51887'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.188470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637264ea67d14e9da3d76d70e761d3b1', 'request_id': 'req-f0274369-c145-47d6-911f-39cf0c5d21e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.200682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c4512401034796abda50f2fda51887'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.202332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-523f0510-345a-45f7-8cf3-ab4d63601031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c4512401034796abda50f2fda51887'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.205506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637264ea67d14e9da3d76d70e761d3b1', 'request_id': 'req-f0274369-c145-47d6-911f-39cf0c5d21e0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.223478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 869/1746] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:47 2022] DELETE /identity/v3/users/f9c4512401034796abda50f2fda51887 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:48.228933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.231577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.232847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.260415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:48.262642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.264202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] The token cache is being invalidate because project 637264ea67d14e9da3d76d70e761d3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:48.264675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.266985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.272250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 878/1747] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:48 2022] DELETE /identity/v3/projects/637264ea67d14e9da3d76d70e761d3b1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:48.274294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:48.276072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.365944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0274369-c145-47d6-911f-39cf0c5d21e0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.376845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:48.391167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 870/1748] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:48.401357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:48.402877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.510163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:48.510854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdde4d-19d4-4772-916d-9bb7e9aa7887 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.511492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.511779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.512057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] PATH_INFO: `/v3/users/596d5d12c7a84cbb807303979fcdce34` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.513899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596d5d12c7a84cbb807303979fcdce34)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:48.516822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:48.529525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:48.531825 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.534338 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.534755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.567895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:48.571719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:48.600700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 871/1749] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:48.601669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:48.606362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.616500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596d5d12c7a84cbb807303979fcdce34'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.629137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596d5d12c7a84cbb807303979fcdce34'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.630090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596d5d12c7a84cbb807303979fcdce34'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.630593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a493e7-d112-4eb2-85ee-4fe821f54cb6 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.635969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:48.636912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:48.637212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:48.637621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] PATH_INFO: `/v3/projects/ab44d9d0e47046f1b834f2a410e73cbd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:48.638811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab44d9d0e47046f1b834f2a410e73cbd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:48.642811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:48.657811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596d5d12c7a84cbb807303979fcdce34'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.658663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596d5d12c7a84cbb807303979fcdce34'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.725962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 879/1750] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:48 2022] DELETE /identity/v3/users/596d5d12c7a84cbb807303979fcdce34 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:48.780797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:48.784672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab44d9d0e47046f1b834f2a410e73cbd', 'request_id': 'req-a98e28da-84c7-4745-abd2-f073737225b9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.795854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab44d9d0e47046f1b834f2a410e73cbd', 'request_id': 'req-a98e28da-84c7-4745-abd2-f073737225b9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.804965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab44d9d0e47046f1b834f2a410e73cbd', 'request_id': 'req-a98e28da-84c7-4745-abd2-f073737225b9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.818942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] The token cache is being invalidate because project ab44d9d0e47046f1b834f2a410e73cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:48.819614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.820756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:48.823294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 872/1751] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:48 2022] DELETE /identity/v3/projects/ab44d9d0e47046f1b834f2a410e73cbd => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:48.830417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:48.832163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:48.919950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e4bca-6c61-4f89-96ef-044e54bcb823 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:48.971689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:49.026955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:49.029568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.030040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.030466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] PATH_INFO: `/v3/users/d613b95312ea474f90daa63aa74fad2a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.032069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d613b95312ea474f90daa63aa74fad2a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:49.034444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:49.131646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd613b95312ea474f90daa63aa74fad2a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.141750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd613b95312ea474f90daa63aa74fad2a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.142551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd613b95312ea474f90daa63aa74fad2a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.157964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd613b95312ea474f90daa63aa74fad2a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.158833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd613b95312ea474f90daa63aa74fad2a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.177593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 880/1752] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:48 2022] DELETE /identity/v3/users/d613b95312ea474f90daa63aa74fad2a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:49.293991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:49.297457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:49.334914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e28da-84c7-4745-abd2-f073737225b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:49.451761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:49.452794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.453139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.453421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] PATH_INFO: `/v3/projects/d1aca28518cc4f5782c74f34479175f1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.454682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1aca28518cc4f5782c74f34479175f1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:49.457507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:49.538207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1aca28518cc4f5782c74f34479175f1', 'request_id': 'req-fedeca21-93a2-4887-8095-d35cddd8e167'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.550096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1aca28518cc4f5782c74f34479175f1', 'request_id': 'req-fedeca21-93a2-4887-8095-d35cddd8e167'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.558326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1aca28518cc4f5782c74f34479175f1', 'request_id': 'req-fedeca21-93a2-4887-8095-d35cddd8e167'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.565271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:49.567164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:49.575607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] The token cache is being invalidate because project d1aca28518cc4f5782c74f34479175f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:49.576314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.576551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedeca21-93a2-4887-8095-d35cddd8e167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.584745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 873/1753] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:49 2022] DELETE /identity/v3/projects/d1aca28518cc4f5782c74f34479175f1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:49.591126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.591417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.595022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.624476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894519b-1658-4811-9ec8-d930ad90bcb5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:49.660301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:49.662821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:49.728662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:49.731686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.732366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.732787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] PATH_INFO: `/v3/projects/a9b3a6a6383f41e59da269deb4c1c460` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.735748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b3a6a6383f41e59da269deb4c1c460)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:49.744199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:49.753125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:49.777217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 874/1754] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:49.780637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:49.782915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:49.805035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49daa07b-dfd7-4bea-b99e-a0da17b2b825 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:49.811894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:49.812974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.813251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.813524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] PATH_INFO: `/v3/projects/cc982fa59ee2420fb64af5c29aef9469` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.814807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc982fa59ee2420fb64af5c29aef9469)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:49.817193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:49.851062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b3a6a6383f41e59da269deb4c1c460', 'request_id': 'req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.863777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b3a6a6383f41e59da269deb4c1c460', 'request_id': 'req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.867801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:49.876198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b3a6a6383f41e59da269deb4c1c460', 'request_id': 'req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.877373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc982fa59ee2420fb64af5c29aef9469', 'request_id': 'req-a7545a48-98b2-47b7-a6e8-a580967b16d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.885740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc982fa59ee2420fb64af5c29aef9469', 'request_id': 'req-a7545a48-98b2-47b7-a6e8-a580967b16d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.893527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc982fa59ee2420fb64af5c29aef9469', 'request_id': 'req-a7545a48-98b2-47b7-a6e8-a580967b16d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.895819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] The token cache is being invalidate because project a9b3a6a6383f41e59da269deb4c1c460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:49.896118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.896791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4a6160a5-25e3-4f19-aec5-51dee0b2ba2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.898750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 881/1755] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:49 2022] DELETE /identity/v3/projects/a9b3a6a6383f41e59da269deb4c1c460 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:49.899851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.900143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.900419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.907211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] The token cache is being invalidate because project cc982fa59ee2420fb64af5c29aef9469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:49.907661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.908495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a7545a48-98b2-47b7-a6e8-a580967b16d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:49.910072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 875/1756] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:49 2022] DELETE /identity/v3/projects/cc982fa59ee2420fb64af5c29aef9469 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:49.911383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:49.911666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:49.911972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:49.919992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:49.921367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:49.941702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:49.942975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.019733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 882/1757] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:50.026978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.027137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 876/1758] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:50.027216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.034690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.034938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cd01-8b09-4706-a5ee-e89595a4909a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.044739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.045244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea753-74dc-4a5e-b5e1-b04f52db833e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.045752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.046163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.046447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] PATH_INFO: `/v3/users/fb709cfd4e9c46759fcb9723717b5c91` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.049829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb709cfd4e9c46759fcb9723717b5c91)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.051728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.051995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.052550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.052865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.053123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.053873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:50.054299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.055971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dac40-99e5-4d5d-b336-42095e3d60a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.078849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 877/1759] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:50.078849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.078849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.078849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.100094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:50.101501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.112332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 878/1760] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:50.115819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.124183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fe910-5bbd-467d-8688-f6b794c759d7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.130345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.131091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.131378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.131656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.132356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.134104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb709cfd4e9c46759fcb9723717b5c91'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.134413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.141780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb709cfd4e9c46759fcb9723717b5c91'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.142496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb709cfd4e9c46759fcb9723717b5c91'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.154939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb709cfd4e9c46759fcb9723717b5c91'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.155630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb709cfd4e9c46759fcb9723717b5c91'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.161450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 879/1761] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:50.162256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.163892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.174287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 883/1762] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:50 2022] DELETE /identity/v3/users/fb709cfd4e9c46759fcb9723717b5c91 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:50.175239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.176623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.229253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:50.256172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d5a2c-32bd-4ce4-b6c3-a96069c6c54d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.270358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:50.275863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93dbd-8f66-4bac-8929-bb1fa8b120e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.282656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.285438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.285438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.285631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.288088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.292966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.337851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 884/1763] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:50.339256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.349838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4f90d-c090-4cf3-8ba3-90b0359f5c72 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.352548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.353478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.353775 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.354191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.355071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:50.355386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.357661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.357661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.357661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.357934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.358053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.358853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.361801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.374550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 880/1764] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:50.374748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 885/1765] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:50.377319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.377440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.388109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554df625-83d0-4802-879f-2889109472c4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.388591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb932-30a7-4314-8885-293c0b20e894 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.393691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.394445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.394821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.394940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.395237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.395454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.396058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.396163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.396257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.396989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.397811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.398991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.430122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 886/1766] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:50.430982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.442505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 881/1767] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:50.443878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.444649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70945b1-dd75-459d-aa7d-2b9c8811b76b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.551581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a586d-c40a-496a-bf8d-b5e982703d15 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.558472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.559192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.559746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.560026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.560727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.563728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.565303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.566358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.566643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.566946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] PATH_INFO: `/v3/projects/f2f4e5a64a054db98d6e6c8acc4bcaca/users/0332632cd49f4b198361a3cd3ad840fd/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.591394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f4e5a64a054db98d6e6c8acc4bcaca, user_id=0332632cd49f4b198361a3cd3ad840fd, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.595906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16faeaa6-9e2e-4621-8be6-c02e31cbf7d5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.610610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 882/1768] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:50.612276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.626899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 887/1769] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:50 2022] PUT /identity/v3/projects/f2f4e5a64a054db98d6e6c8acc4bcaca/users/0332632cd49f4b198361a3cd3ad840fd/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:50.635895 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11651eea-5675-4038-ac86-44f8c71b6a07 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.649380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.649830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.650119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.719837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] MFA Rules not processed for user `0332632cd49f4b198361a3cd3ad840fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:50.721119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.722564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:50.740315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.740991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.741280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.741564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.742421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.744077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.754011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 883/1770] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:50.754856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.756507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.769435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ff48-fec3-4061-bb11-7fd3444ab145 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.774429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.775435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.775729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.776011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] PATH_INFO: `/v3/projects/7d72f2d1dc094c2fa003f800789ea450` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.777194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d72f2d1dc094c2fa003f800789ea450)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.779121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.794012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 888/1771] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:50.794714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.805054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07dafa-f134-433a-9f4e-3c979b6c0971 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.810235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.811071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.811353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.811640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.812395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.814006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.824085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 889/1772] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:50.824699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.834339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab63c2d-ec48-4485-9316-bfc2eca10b43 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.836472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d72f2d1dc094c2fa003f800789ea450', 'request_id': 'req-8e88ee8c-12a7-42fc-acb3-f460c79b5620'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.844966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d72f2d1dc094c2fa003f800789ea450', 'request_id': 'req-8e88ee8c-12a7-42fc-acb3-f460c79b5620'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.853169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d72f2d1dc094c2fa003f800789ea450', 'request_id': 'req-8e88ee8c-12a7-42fc-acb3-f460c79b5620'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.859060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:50.864581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] The token cache is being invalidate because project 7d72f2d1dc094c2fa003f800789ea450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:50.864828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.865558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:50.867223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 884/1773] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:50 2022] DELETE /identity/v3/projects/7d72f2d1dc094c2fa003f800789ea450 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:50.868071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:50.869383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.897687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.898345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.898632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.898963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.899645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:50.901544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4311b4-d56a-4766-83e2-8bc12590404c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:50.926542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 890/1774] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:50.928102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.928397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.928673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:50.934632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88ee8c-12a7-42fc-acb3-f460c79b5620 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:50.947581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:50.948790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:50.991202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:50.992084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:50.992367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:50.992645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] PATH_INFO: `/v3/projects/c2afbdee29794a3285d6eaf1bdd3c3bb/users/04942747ea4e4ed4a17dce9b954caadc/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.007700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2afbdee29794a3285d6eaf1bdd3c3bb, user_id=04942747ea4e4ed4a17dce9b954caadc, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.008840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 891/1775] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:51.009663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.010342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ee9c7-0ee3-41de-9940-b4902c5d321b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.011063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.019534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:51.023823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9660f1-4492-4938-897d-d2c9e97460a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.026297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 885/1776] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:50 2022] PUT /identity/v3/projects/c2afbdee29794a3285d6eaf1bdd3c3bb/users/04942747ea4e4ed4a17dce9b954caadc/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:51.029995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314647f-9b0e-472e-bf4e-f885e3d40ffc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.029995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314647f-9b0e-472e-bf4e-f885e3d40ffc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.029995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314647f-9b0e-472e-bf4e-f885e3d40ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.045914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5314647f-9b0e-472e-bf4e-f885e3d40ffc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:51.047182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5314647f-9b0e-472e-bf4e-f885e3d40ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.082494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.083196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.083484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.083865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.084553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.086468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.105887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 886/1777] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:51.107193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375781-9580-465c-8caa-968515565336 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.107483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375781-9580-465c-8caa-968515565336 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.107656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 892/1778] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:51.108031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85375781-9580-465c-8caa-968515565336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.112728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.122571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b75a7-8ebb-4f9c-a0ce-3659cbb3c3e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.126007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-85375781-9580-465c-8caa-968515565336 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:51.127225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-85375781-9580-465c-8caa-968515565336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.127950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.128620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.128908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.129186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.129918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.131624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.135053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 887/1779] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:51.135880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85375781-9580-465c-8caa-968515565336 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.143224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 893/1780] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:51.147032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.150497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85375781-9580-465c-8caa-968515565336 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.155343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.155891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.156701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.156975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.157541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38137587-6517-4772-b531-af4bb545abf5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.157697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:51.158002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.159618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.162168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.163084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.163370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.163648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] PATH_INFO: `/v3/projects/5c5f28f161544577949b27fd328567ca/users/88e39b1870364a0586c2517a2a10c0f2/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.173207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 888/1781] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:51.174124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.176659 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.181174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5f28f161544577949b27fd328567ca, user_id=88e39b1870364a0586c2517a2a10c0f2, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.183916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.199623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 894/1782] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:51 2022] PUT /identity/v3/projects/5c5f28f161544577949b27fd328567ca/users/88e39b1870364a0586c2517a2a10c0f2/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:08:51.200419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.209118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:51.245559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d90d2-41fb-48d9-b052-31f059fbd958 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.293674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27c09b-4f29-4626-b74a-8a9101a7c678 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.300952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.301718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.302101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.302444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.303254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.305537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.307635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.308286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.308564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.308835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.318533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.320109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.331278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 889/1783] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:51.332033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.333948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 895/1784] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/projects => generated 399 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:51.338084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.350350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd041b-70b1-41bc-86fb-708402942efa service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.407407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:51.436808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2995aa2-b853-45e1-9c0c-225c7ad3b0dc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.441379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.442078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.442218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.442370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.442829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.443091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.443437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.443558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.443888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.444709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.445377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.447012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.462444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 890/1785] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:51.466344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.475928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 896/1786] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:51.480456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.483101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657aa86-0930-45ba-b4f1-eae456f0a78d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.490320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.491682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.492078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.492483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] PATH_INFO: `/v3/projects/5c5f28f161544577949b27fd328567ca/users/88e39b1870364a0586c2517a2a10c0f2/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.495264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5f28f161544577949b27fd328567ca, user_id=88e39b1870364a0586c2517a2a10c0f2, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.498490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981caa19-11d7-456f-8cab-576e790955c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.513487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 891/1787] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:51 2022] PUT /identity/v3/projects/5c5f28f161544577949b27fd328567ca/users/88e39b1870364a0586c2517a2a10c0f2/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:51.517116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8909f3-d17e-4a16-9a41-4faed581557c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.524939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.525541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.525838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.564501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] MFA Rules not processed for user `88e39b1870364a0586c2517a2a10c0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:51.565925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.663199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.664076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.664268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.664566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.666396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.668195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.668627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 892/1788] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:51.689742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.693950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:51.728768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 897/1789] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:51.732548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.745634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984797f-bac7-4eec-9994-ee6a27287a5a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.751610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.752629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.753189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.753750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.755100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.757204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.781658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 898/1790] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:51.783800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:51.797495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7f21-fb34-44b6-89d2-82daecd98876 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.803870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.805095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.805737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.806281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] PATH_INFO: `/v3/projects/a384f7cd401747e1969dae538aae937f/users/be79aaa48e0c487ca2ab0a268333c84b/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.809129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:51.816482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59344251-80d9-480a-a89d-02c9c045ea24 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:51.820719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:51.821455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.821865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.822219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.827966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a384f7cd401747e1969dae538aae937f, user_id=be79aaa48e0c487ca2ab0a268333c84b, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.831400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.833514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:51.835160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c72a0-646e-4b4a-a6ae-14363eac7178 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:51.851758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 893/1791] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:51.875435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 899/1792] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:51 2022] PUT /identity/v3/projects/a384f7cd401747e1969dae538aae937f/users/be79aaa48e0c487ca2ab0a268333c84b/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:51.881099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:51.881628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:51.882100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:51.931039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] MFA Rules not processed for user `be79aaa48e0c487ca2ab0a268333c84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:51.932295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:52.001826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:52.067400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 894/1793] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:52.072080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.262321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3dcff-b186-4c58-9dca-e53472714f81 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.268026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.268930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.269222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.269505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.270221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.272202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.322066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 900/1794] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/projects => generated 399 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:52.326466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.338312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96746ec4-1f56-413f-90a3-3aefa79e1874 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.486692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.487854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.488143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.488419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.489129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.494470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.543697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 895/1795] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/users => generated 306 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:52.548007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.571526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd94bb-9b21-4635-aafe-7612bf5a4f22 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.577095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.578452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.578734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.579170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.579945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.581581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa70d3f-b294-42e8-b78e-cc41c92ed3bd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.594191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 901/1796] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:52.598372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.609916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db2498-c6fe-48c5-b84e-2c364ea0e22d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.615271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.616219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.616527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.616801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] PATH_INFO: `/v3/projects/875966eb97ce40b2b1743fc9d1157553/users/ac67ca39faf14c59a55eb7799f40a711/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.638889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875966eb97ce40b2b1743fc9d1157553, user_id=ac67ca39faf14c59a55eb7799f40a711, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.642572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.674401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 896/1797] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:52 2022] PUT /identity/v3/projects/875966eb97ce40b2b1743fc9d1157553/users/ac67ca39faf14c59a55eb7799f40a711/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:52.675066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.675174 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.675806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.675806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.705674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] MFA Rules not processed for user `04942747ea4e4ed4a17dce9b954caadc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:52.707051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:52.774678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb843e-0ec8-4d49-af0f-7fd593548ac7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.780770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.781535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.781827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.782355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.782913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.784676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.805305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 897/1798] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:52.811951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.816093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 902/1799] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:52.817753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.835117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb300206-f310-4bd2-a3a4-9f4128745f21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.840998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.842462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.842462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.842633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.843359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.845785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.847483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c27865-1b0d-4d97-a364-e9b516248286 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:52.854958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:52.856596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.857093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.857550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] PATH_INFO: `/v3/projects/875966eb97ce40b2b1743fc9d1157553/users/ac67ca39faf14c59a55eb7799f40a711/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.861902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875966eb97ce40b2b1743fc9d1157553, user_id=ac67ca39faf14c59a55eb7799f40a711, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:52.865697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917da66-265a-4fca-aaa4-7bd3449ce433 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:52.880930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 903/1800] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:52.881844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 898/1801] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:52 2022] PUT /identity/v3/projects/875966eb97ce40b2b1743fc9d1157553/users/ac67ca39faf14c59a55eb7799f40a711/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:52.884664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:52.893467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:52.893764 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:52.894110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:52.929632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:52.953359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] MFA Rules not processed for user `ac67ca39faf14c59a55eb7799f40a711`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:52.954747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:52.981058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:53.008716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6905df-6a24-4cd9-a9b5-65793383e4fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.015658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.016539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.016948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.017416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.018218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.022579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.067906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 904/1802] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:53.072154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.084357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392decb7-a25c-4461-ae80-6a219a834bc4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.089720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.090473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.090885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.091265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.092282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.094855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.112080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 905/1803] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:08:53.113320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.126518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8facb2-145d-40e4-9abd-4a6bdcbb380a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.129940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 899/1804] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:53.143707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.147707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.148005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.148293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] PATH_INFO: `/v3/projects/50eaa81ab24341e7ab8054f311c0a873/users/bfb77a2726f542f88fa1f123c1b42257/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.157917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.160847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:53.178309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50eaa81ab24341e7ab8054f311c0a873, user_id=bfb77a2726f542f88fa1f123c1b42257, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.187458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.208526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 906/1805] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:53 2022] PUT /identity/v3/projects/50eaa81ab24341e7ab8054f311c0a873/users/bfb77a2726f542f88fa1f123c1b42257/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:53.219703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.319865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9d24-7aa4-4ede-82c0-f6966924d760 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.327202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.328997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.329511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.329895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.330880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.332738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.333559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cf115-dc8f-44d2-9b4e-3a116342d920 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.360792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 907/1806] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:53.361703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.392120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12e20d-0070-4a1c-b44b-e543d1ee742c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.400794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.401886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.402170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.402442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] PATH_INFO: `/v3/projects/50eaa81ab24341e7ab8054f311c0a873/users/bfb77a2726f542f88fa1f123c1b42257/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.413962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50eaa81ab24341e7ab8054f311c0a873, user_id=bfb77a2726f542f88fa1f123c1b42257, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.428063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.428710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.429439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.429731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.430012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.469110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 908/1807] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:53 2022] PUT /identity/v3/projects/50eaa81ab24341e7ab8054f311c0a873/users/bfb77a2726f542f88fa1f123c1b42257/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:53.529712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.531254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.548599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 900/1808] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:53.669447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:53.788958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d7fd-89d6-45f7-9c9d-4744b002a977 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:53.794455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:53.795436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:53.796940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:53.796940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:53.827583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:53.935206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:53.937592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e977a9-acd1-4e5b-adbb-512afb836598 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:53.949573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 909/1809] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:54.124653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:54.135765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c71a4-ce2d-4865-b15e-37c54c0c6320 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:54.154480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:54.221367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:54.222285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.222556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.222892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] PATH_INFO: `/v3/users/0332632cd49f4b198361a3cd3ad840fd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:54.224339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0332632cd49f4b198361a3cd3ad840fd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:54.226140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:54.301571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0332632cd49f4b198361a3cd3ad840fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:54.314253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0332632cd49f4b198361a3cd3ad840fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:54.315402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0332632cd49f4b198361a3cd3ad840fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:54.348433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0332632cd49f4b198361a3cd3ad840fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:54.349482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6efe1d4b-56d6-439a-891f-a498a31f5e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0332632cd49f4b198361a3cd3ad840fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:54.369554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 901/1810] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:54 2022] DELETE /identity/v3/users/0332632cd49f4b198361a3cd3ad840fd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:54.371671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.371963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.372305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:54.391535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:54.393747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.467630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.467922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.468990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:54.474161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 910/1811] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:54.489531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:54.493260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.506802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] MFA Rules not processed for user `bfb77a2726f542f88fa1f123c1b42257`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:54.507947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.588372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab642ff-629a-4e2b-a443-c177ac26707d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:54.593010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 902/1812] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:08:54.599630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:54.600619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.601062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.601490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:54.606004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:54.607462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.613339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:54.615277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:54.639882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 911/1813] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:54.646990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:54.702996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf181-f2f2-4dda-a802-4b1bc4e3b22a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:54.708038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:54.708800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.710004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.710700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:54.714981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.814003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:54.815511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:54.829960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 903/1814] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:54.958585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:54.960214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:54.990859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a049871-0f84-4cd1-9d9b-0a36fb09d5da service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:54.996844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:54.998269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:54.998577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:54.998969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] PATH_INFO: `/v3/projects/f2f4e5a64a054db98d6e6c8acc4bcaca` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.001181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f4e5a64a054db98d6e6c8acc4bcaca)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.003311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:55.052893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f4e5a64a054db98d6e6c8acc4bcaca', 'request_id': 'req-d70d80ff-fa52-403e-8891-132118beef19'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.059583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f4e5a64a054db98d6e6c8acc4bcaca', 'request_id': 'req-d70d80ff-fa52-403e-8891-132118beef19'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.061338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:55.068227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f4e5a64a054db98d6e6c8acc4bcaca', 'request_id': 'req-d70d80ff-fa52-403e-8891-132118beef19'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.084151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] The token cache is being invalidate because project f2f4e5a64a054db98d6e6c8acc4bcaca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:55.084743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.085464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.087411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 912/1815] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:54 2022] DELETE /identity/v3/projects/f2f4e5a64a054db98d6e6c8acc4bcaca => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:55.091076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:55.102585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0874-8923-4dfd-858e-ef53539f28d9 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:55.169940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:55.170966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.171253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.172097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] PATH_INFO: `/v3/users/88e39b1870364a0586c2517a2a10c0f2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.173468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e39b1870364a0586c2517a2a10c0f2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.175407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:55.222482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e39b1870364a0586c2517a2a10c0f2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.228885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e39b1870364a0586c2517a2a10c0f2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.229559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e39b1870364a0586c2517a2a10c0f2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.241974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e39b1870364a0586c2517a2a10c0f2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.242628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-df0802a4-1037-4b3d-b088-7cbe2b686bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e39b1870364a0586c2517a2a10c0f2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.247779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:55.256541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 904/1816] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:08:55 2022] DELETE /identity/v3/users/88e39b1870364a0586c2517a2a10c0f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:55.530597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:55.532658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:55.551138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d80ff-fa52-403e-8891-132118beef19 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:55.607672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:55.608455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.608737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.609669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] PATH_INFO: `/v3/projects/5c5f28f161544577949b27fd328567ca` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.610863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5f28f161544577949b27fd328567ca)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.612766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:55.655525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5f28f161544577949b27fd328567ca', 'request_id': 'req-7f70ef97-2ab0-41a8-b007-48cfe2bca010'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.661874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5f28f161544577949b27fd328567ca', 'request_id': 'req-7f70ef97-2ab0-41a8-b007-48cfe2bca010'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.668646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5f28f161544577949b27fd328567ca', 'request_id': 'req-7f70ef97-2ab0-41a8-b007-48cfe2bca010'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.681281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] The token cache is being invalidate because project 5c5f28f161544577949b27fd328567ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:08:55.681570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.682219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f70ef97-2ab0-41a8-b007-48cfe2bca010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:08:55.684014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 913/1817] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:08:55 2022] DELETE /identity/v3/projects/5c5f28f161544577949b27fd328567ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:55.689564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.690019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.690114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.708977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:55.710177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:55.773613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 905/1818] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:55.781106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.781402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.781684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.802480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:55.803786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:55.812869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 914/1819] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:55.816929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:55.828150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952bc5e-142b-4d79-995d-2264b3c89581 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:55.839187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:55.839992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.840520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.841023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.841933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:08:55.842275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.843953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:55.858603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 906/1820] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:08:55.863348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:55.872889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d499f5-0446-4e97-a7ef-786881b099ba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:55.878763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:55.880473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.880823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.881103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.881809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.883777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:55.914123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 915/1821] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:55 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:55.915312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:55.926279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9059b-b23c-465e-a6bf-92fe8116f451 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:55.985593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:55.986273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:55.986561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:55.986876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:55.987622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:55.989581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.012177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 907/1822] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:55 2022] POST /identity/v3/users => generated 305 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:56.016830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.027634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de478db-ab00-4ec7-85ef-ec58f94f78ce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.032234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.032899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.033179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.033453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.034254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.035895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.047354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 916/1823] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:56.051515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.063075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd6a8d-5cd9-4e32-9478-c238b3261ef1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.067806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.068479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.068749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.069027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.069770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.071521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c42fa7-b77e-4562-8831-5e90452e6590 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.085494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 908/1824] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:56.087469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.098531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7eae-718d-4b9b-b475-b786a89b7549 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.104976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.106075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.106435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.106814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] PATH_INFO: `/v3/projects/4ec5c0e7f60d40e1b44b906d242f23fd/users/33604cd09c17488bbec124352b97712d/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.133813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec5c0e7f60d40e1b44b906d242f23fd, user_id=33604cd09c17488bbec124352b97712d, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.136635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.152627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 917/1825] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:56 2022] PUT /identity/v3/projects/4ec5c0e7f60d40e1b44b906d242f23fd/users/33604cd09c17488bbec124352b97712d/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:56.164377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484411f-f783-4078-a193-34717f52bd1e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.164689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484411f-f783-4078-a193-34717f52bd1e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.164966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484411f-f783-4078-a193-34717f52bd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.207108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3484411f-f783-4078-a193-34717f52bd1e None None] MFA Rules not processed for user `33604cd09c17488bbec124352b97712d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:56.207108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3484411f-f783-4078-a193-34717f52bd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:56.250016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:56.267251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 909/1826] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:56.271735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.282078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:56.334363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a9d2e-8020-4084-a385-2d8b9b1e9b8c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.339016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.339692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.339977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.340253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.340941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.342855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.368640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 918/1827] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:08:56 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:08:56.372510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484411f-f783-4078-a193-34717f52bd1e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.383258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484411f-f783-4078-a193-34717f52bd1e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.444548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.445219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.445511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.445795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.446475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.448410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.467751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 910/1828] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:08:56 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:08:56.472216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.482468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f4781-8233-4d7e-879e-45f008dc8013 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.488251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.488397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.488654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.488941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.489652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.491286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.502120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 919/1829] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:56.506583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.518260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a75911-5762-42ae-8956-3605b60f713b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.523274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.524354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.524639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.524917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] PATH_INFO: `/v3/projects/6702288e65ed43e0b0ec4be5854053d1/users/8ffb84aeba5b480e94d47185721c5bea/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.542194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6702288e65ed43e0b0ec4be5854053d1, user_id=8ffb84aeba5b480e94d47185721c5bea, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.544874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.557721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 911/1830] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:56 2022] PUT /identity/v3/projects/6702288e65ed43e0b0ec4be5854053d1/users/8ffb84aeba5b480e94d47185721c5bea/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:56.561843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.571798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c9f40-1de5-4757-888a-866a477e9782 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.633619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.634297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.634729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.634976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.635606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.637890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f0876-b57e-4a83-ba60-c8e354b6758e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.650668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 920/1831] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:08:56.655011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.672997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f280402-300f-48e9-8812-5a8dccd19abe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.677887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.678814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.679082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.679357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] PATH_INFO: `/v3/projects/6702288e65ed43e0b0ec4be5854053d1/users/8ffb84aeba5b480e94d47185721c5bea/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.681865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6702288e65ed43e0b0ec4be5854053d1, user_id=8ffb84aeba5b480e94d47185721c5bea, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:56.684932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:56.701293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 912/1832] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:08:56 2022] PUT /identity/v3/projects/6702288e65ed43e0b0ec4be5854053d1/users/8ffb84aeba5b480e94d47185721c5bea/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:08:56.707623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.707976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.709279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:56.739313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] MFA Rules not processed for user `8ffb84aeba5b480e94d47185721c5bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:56.741025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:56.859515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 921/1833] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:08:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:56.866722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:56.869066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:56.973497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b2e7d-fa96-4580-908e-c48a3040a894 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:56.980241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:56.980924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:56.981214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:56.981495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:57.002691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:57.004164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:57.034429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 913/1834] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:57.324910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:57.328205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:57.342250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:57.350591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cace61-a51a-412a-a867-aa4f9ac664ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:57.363220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:57.365324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:57.365601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:57.366397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:57.367075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:57.370703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:57.395209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ec48-1782-401c-910a-8f2820a68cb8 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:57.413016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:57.413896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:57.417150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:57.419190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:57.503724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:57.505204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a63ab5-6c5d-4ab0-9fb9-b5216497e5dd service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:57.516985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 922/1835] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:08:57.565845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:08:57.596632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:57.598229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7adda-ff55-4f3f-8f2e-b38e38ec4ea8 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:57.609239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 914/1836] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:57 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:08:58.784906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:58.786748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:58.787324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:58.789931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 923/1837] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:08:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:08:58.990615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:58.991127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:58.991384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:59.025684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:08:59.027287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:59.271622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 915/1838] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:08:59.393825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:08:59.396425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:08:59.426782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ef907-5fc8-4b99-bf92-8a5197f308c9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:08:59.445024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:08:59.446011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:08:59.446519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:08:59.447200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:08:59.473992 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:08:59.477162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d70beb-fc49-40ba-aa06-090ec3d01b16 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:08:59.496475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 924/1839] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:09.242262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=38 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:09.487906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=34 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:11.868278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:11.869889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:11.957676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fcea1-76ca-405e-b1f3-d04db7753225 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:11.963848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:11.964478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:11.964837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:11.965326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] PATH_INFO: `/v3/users/be79aaa48e0c487ca2ab0a268333c84b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:11.966982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be79aaa48e0c487ca2ab0a268333c84b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:11.970003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:12.036061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be79aaa48e0c487ca2ab0a268333c84b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:12.045729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be79aaa48e0c487ca2ab0a268333c84b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:12.046410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be79aaa48e0c487ca2ab0a268333c84b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:12.062917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be79aaa48e0c487ca2ab0a268333c84b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:12.063701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be79aaa48e0c487ca2ab0a268333c84b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:12.076471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 916/1840] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:11 2022] DELETE /identity/v3/users/be79aaa48e0c487ca2ab0a268333c84b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:12.081171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:12.081671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:12.081863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:12.100178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:12.101763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:12.167446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 925/1841] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:12.177922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:12.179496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:12.251523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab644f0-27ba-4f52-9aa2-25f8c6d34142 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:12.255438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:12.256312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:12.256437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:12.256668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:12.269091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:12.270695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:12.288719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 917/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:12.937367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:12.939720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:12.952977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38721059-0f82-4694-a670-73311a5c5cc5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:12.956827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:12.957881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:12.972604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:12.973540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:12.973799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:12.974119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] PATH_INFO: `/v3/users/33604cd09c17488bbec124352b97712d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:12.978430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8fbcae-e7b1-4493-9bcc-976576d27ad9 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:12.979618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33604cd09c17488bbec124352b97712d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:12.981574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:12.989172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:12.990249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:12.990456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:12.990706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] PATH_INFO: `/v3/users/84537e9b76d248d9b1e48d3f29c373b6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.003728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84537e9b76d248d9b1e48d3f29c373b6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:13.005876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:13.036142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:13.068710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33604cd09c17488bbec124352b97712d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.075254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84537e9b76d248d9b1e48d3f29c373b6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.078857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33604cd09c17488bbec124352b97712d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.080448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33604cd09c17488bbec124352b97712d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.090279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84537e9b76d248d9b1e48d3f29c373b6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.091526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84537e9b76d248d9b1e48d3f29c373b6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.101250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33604cd09c17488bbec124352b97712d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.102238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33604cd09c17488bbec124352b97712d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.121299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 926/1843] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:12 2022] DELETE /identity/v3/users/33604cd09c17488bbec124352b97712d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:13.122636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:13.124776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.148931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84537e9b76d248d9b1e48d3f29c373b6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.150255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24ef7229-5c2f-4764-946b-27698d5c7dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84537e9b76d248d9b1e48d3f29c373b6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.203038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 918/1844] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:12 2022] DELETE /identity/v3/users/84537e9b76d248d9b1e48d3f29c373b6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:13.203038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f91b75-4a01-477e-8238-b0b253e4fa02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.203038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f91b75-4a01-477e-8238-b0b253e4fa02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.203038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f91b75-4a01-477e-8238-b0b253e4fa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.218971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f13dc2-e7b1-44d6-b295-164c056c63f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:13.237460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:13.243893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-22f91b75-4a01-477e-8238-b0b253e4fa02 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:13.249205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22f91b75-4a01-477e-8238-b0b253e4fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.293133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:13.294071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.294357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.294631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] PATH_INFO: `/v3/projects/a384f7cd401747e1969dae538aae937f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.295895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a384f7cd401747e1969dae538aae937f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:13.297980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:13.342730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 919/1845] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:13.345653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.345653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.345653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.366555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:13.368050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.372635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a384f7cd401747e1969dae538aae937f', 'request_id': 'req-5045e0a7-e7df-493f-884d-c60a4f2f8a00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.387585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a384f7cd401747e1969dae538aae937f', 'request_id': 'req-5045e0a7-e7df-493f-884d-c60a4f2f8a00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.400062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a384f7cd401747e1969dae538aae937f', 'request_id': 'req-5045e0a7-e7df-493f-884d-c60a4f2f8a00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.420595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] The token cache is being invalidate because project a384f7cd401747e1969dae538aae937f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:13.421569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.422758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.428377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 927/1846] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:13 2022] DELETE /identity/v3/projects/a384f7cd401747e1969dae538aae937f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:13.429621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:13.431327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.477988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 920/1847] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:13.479296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:13.480854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.494935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84893b-21fd-4a84-bae0-1c08308d2575 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:13.503196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:13.504679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.507103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.507641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] PATH_INFO: `/v3/users/ac67ca39faf14c59a55eb7799f40a711` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.512243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac67ca39faf14c59a55eb7799f40a711)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:13.515218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:13.532068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e0a7-e7df-493f-884d-c60a4f2f8a00 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:13.537842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:13.538500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.538804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.539743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.541623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.559577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:13.561432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:13.567482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:13.575363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 928/1848] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:13.576574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:13.577919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.589738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac67ca39faf14c59a55eb7799f40a711'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.602463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259607e-e097-4ca5-a169-4e98e2122e4f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:13.603576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac67ca39faf14c59a55eb7799f40a711'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.604333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac67ca39faf14c59a55eb7799f40a711'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.621716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:13.622720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:13.623223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:13.623663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:13.627594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:13.635525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac67ca39faf14c59a55eb7799f40a711'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.636517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac67ca39faf14c59a55eb7799f40a711'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:13.644432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:13.646664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:13.654303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 921/1849] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:13 2022] DELETE /identity/v3/users/ac67ca39faf14c59a55eb7799f40a711 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:13.671814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 929/1850] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:14.176195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:14.177819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:14.271737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:14.273176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:14.291551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada5759-72d4-4f1f-914c-65768cd78a34 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:14.307657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:14.308613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:14.308823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:14.309085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] PATH_INFO: `/v3/projects/ef58452af01448e4b56d2f82a483937b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:14.312151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef58452af01448e4b56d2f82a483937b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:14.318247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:14.328440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f899-72f0-4ce2-88f6-3d75c2282663 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:14.351692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:14.352618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:14.352886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:14.354563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] PATH_INFO: `/v3/projects/4ec5c0e7f60d40e1b44b906d242f23fd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:14.356160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec5c0e7f60d40e1b44b906d242f23fd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:14.358231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:14.406840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:14.435212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef58452af01448e4b56d2f82a483937b', 'request_id': 'req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.455669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef58452af01448e4b56d2f82a483937b', 'request_id': 'req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.484120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef58452af01448e4b56d2f82a483937b', 'request_id': 'req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.494711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec5c0e7f60d40e1b44b906d242f23fd', 'request_id': 'req-c37eaf74-dcd5-4248-a93d-74f8849b36bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.506704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] The token cache is being invalidate because project ef58452af01448e4b56d2f82a483937b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:14.507040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.508652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.514469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 930/1851] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:14 2022] DELETE /identity/v3/projects/ef58452af01448e4b56d2f82a483937b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:14.515970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec5c0e7f60d40e1b44b906d242f23fd', 'request_id': 'req-c37eaf74-dcd5-4248-a93d-74f8849b36bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.519224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:14.532642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:14.537166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec5c0e7f60d40e1b44b906d242f23fd', 'request_id': 'req-c37eaf74-dcd5-4248-a93d-74f8849b36bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.581029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] The token cache is being invalidate because project 4ec5c0e7f60d40e1b44b906d242f23fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:14.581985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.583208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.589860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 922/1852] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:14 2022] DELETE /identity/v3/projects/4ec5c0e7f60d40e1b44b906d242f23fd => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:14.591358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:14.596392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:14.678685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d2bc8-d6a4-43cd-9cea-5be4fb84c694 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:14.691828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:14.719583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eaf74-dcd5-4248-a93d-74f8849b36bd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:14.725456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:14.726738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:14.727155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:14.727462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] PATH_INFO: `/v3/users/8b37d5de39e54fd79988554a0ea241e4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:14.728798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b37d5de39e54fd79988554a0ea241e4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:14.730846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:14.743760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:14.744772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:14.745155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:14.745633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] PATH_INFO: `/v3/projects/875966eb97ce40b2b1743fc9d1157553` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:14.746801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=875966eb97ce40b2b1743fc9d1157553)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:14.748786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:14.800255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b37d5de39e54fd79988554a0ea241e4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.809961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b37d5de39e54fd79988554a0ea241e4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.810350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875966eb97ce40b2b1743fc9d1157553', 'request_id': 'req-772d06e9-c213-44ad-89a9-e5be4b244c8b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.810626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b37d5de39e54fd79988554a0ea241e4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.820507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875966eb97ce40b2b1743fc9d1157553', 'request_id': 'req-772d06e9-c213-44ad-89a9-e5be4b244c8b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.828807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b37d5de39e54fd79988554a0ea241e4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.830192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b37d5de39e54fd79988554a0ea241e4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.835272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875966eb97ce40b2b1743fc9d1157553', 'request_id': 'req-772d06e9-c213-44ad-89a9-e5be4b244c8b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.850286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:14.862804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 923/1853] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:14 2022] DELETE /identity/v3/users/8b37d5de39e54fd79988554a0ea241e4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:14.862804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:14.863234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] The token cache is being invalidate because project 875966eb97ce40b2b1743fc9d1157553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:14.863234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.863395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:14.874367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-772d06e9-c213-44ad-89a9-e5be4b244c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:14.883107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 931/1854] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:14 2022] DELETE /identity/v3/projects/875966eb97ce40b2b1743fc9d1157553 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:14.888725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510624c-adfd-4bb2-912a-e6bf747998b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:14.890018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510624c-adfd-4bb2-912a-e6bf747998b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:14.891452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510624c-adfd-4bb2-912a-e6bf747998b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:14.944041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b510624c-adfd-4bb2-912a-e6bf747998b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:14.945358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b510624c-adfd-4bb2-912a-e6bf747998b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.064123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc4338-5b6d-435b-9929-5631e9fe7d82 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.075875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.076781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.077065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.095254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] PATH_INFO: `/v3/users/8ffb84aeba5b480e94d47185721c5bea` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.097027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffb84aeba5b480e94d47185721c5bea)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.100527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.111092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 932/1855] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:15.113069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.113426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.113943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.138696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:15.140437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.152323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 933/1856] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:15.161745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.183132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31c0d-c1e1-4753-95b6-0e4192c1ca5c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.187701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffb84aeba5b480e94d47185721c5bea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.195701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.196578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.197054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.198544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.199162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffb84aeba5b480e94d47185721c5bea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.200210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffb84aeba5b480e94d47185721c5bea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.203031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:15.203626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.205553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.206241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:15.218525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffb84aeba5b480e94d47185721c5bea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.219571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffb84aeba5b480e94d47185721c5bea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.231370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 934/1857] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:15.235217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.236164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 924/1858] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:14 2022] DELETE /identity/v3/users/8ffb84aeba5b480e94d47185721c5bea => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:15.240560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.266185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133c57-1bc0-4e41-8097-b41ec6c7d5bc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.276742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.278608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.361491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.362481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.362891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.363294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.364313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.366597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.389004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d3de5-c97c-485b-b5a4-59946029594c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.395948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.397361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.397652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.397930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] PATH_INFO: `/v3/projects/438bf2aaaa7a47daa4f3dd479766898f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.399571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=438bf2aaaa7a47daa4f3dd479766898f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.402315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.407679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 935/1859] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:15.411292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.472263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438bf2aaaa7a47daa4f3dd479766898f', 'request_id': 'req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.481937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438bf2aaaa7a47daa4f3dd479766898f', 'request_id': 'req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.491284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb282d-e864-4819-b5e4-5cc5a32e8f20 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.491454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438bf2aaaa7a47daa4f3dd479766898f', 'request_id': 'req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.508390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:15.517047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] The token cache is being invalidate because project 438bf2aaaa7a47daa4f3dd479766898f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:15.517229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.517935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e51191f0-feb5-42ff-ad1b-dcc07dc5f9bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.520521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 925/1860] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:15 2022] DELETE /identity/v3/projects/438bf2aaaa7a47daa4f3dd479766898f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:15.525096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.525386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.526037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.548909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:15.551881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.573585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.574905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.575403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.575915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.577677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.580001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.611842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 936/1861] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:15.611842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.611842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.626845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 926/1862] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:15.626845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.627064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb588-2a05-40cd-a9ea-5b8fa738f3e4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.629295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.631098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.632145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.632529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.632905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] PATH_INFO: `/v3/projects/6702288e65ed43e0b0ec4be5854053d1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.634211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6702288e65ed43e0b0ec4be5854053d1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.636270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.643932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993973a-2ca2-4c4d-add0-0f5f54fb1596 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.649468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.650135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.650562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.650884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.652176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.653822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95348bb0-9157-476d-be82-e2a13ebbcf26 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.661556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:15.669803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 927/1863] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:15.669803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da27210-5ff3-43b9-b71b-7d3e323b9cf9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.669803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da27210-5ff3-43b9-b71b-7d3e323b9cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.679435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da27210-5ff3-43b9-b71b-7d3e323b9cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.708756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6702288e65ed43e0b0ec4be5854053d1', 'request_id': 'req-9b4f1a57-4a19-4326-95d4-f627166705d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.716397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7da27210-5ff3-43b9-b71b-7d3e323b9cf9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:15.717898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7da27210-5ff3-43b9-b71b-7d3e323b9cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.718088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6702288e65ed43e0b0ec4be5854053d1', 'request_id': 'req-9b4f1a57-4a19-4326-95d4-f627166705d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.730920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6702288e65ed43e0b0ec4be5854053d1', 'request_id': 'req-9b4f1a57-4a19-4326-95d4-f627166705d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.744460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] The token cache is being invalidate because project 6702288e65ed43e0b0ec4be5854053d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:15.745897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.745897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:15.748197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 937/1864] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:15 2022] DELETE /identity/v3/projects/6702288e65ed43e0b0ec4be5854053d1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:15.749002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.750665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.822674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 928/1865] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:15.827235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.827592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.827999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.842022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f1a57-4a19-4326-95d4-f627166705d5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.847081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:15.848026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.848294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.848867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.849152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.849604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.850398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.852602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.868334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 929/1866] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:15.868334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.868489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 938/1867] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:15.868489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.871461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07baae-14a3-47b9-b597-9dc1d34ff154 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.879671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.880631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.880926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.881204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.882182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:15.882495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.882894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c15831-b10f-4441-9c1c-e16393e449bd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.884317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b08098-f76d-4201-a81c-168601d5d3b2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.888846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:15.889745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.890037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.890481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] PATH_INFO: `/v3/projects/3960585bb5274a34b138638da4f508a5/users/c16d64a16a094243abb17d781dbfef42/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.900900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 930/1868] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:15.904571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:15.904571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:15.904571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:15.917940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3960585bb5274a34b138638da4f508a5, user_id=c16d64a16a094243abb17d781dbfef42, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:15.924868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:15.926497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:15.927887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:15.938647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:15.940307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 931/1869] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:15.945617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:15.953317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd7848-a31c-4959-b3db-5bc3b6e3b587 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:15.959901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 939/1870] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:15 2022] PUT /identity/v3/projects/3960585bb5274a34b138638da4f508a5/users/c16d64a16a094243abb17d781dbfef42/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:15.961171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.041719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.042474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.042755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.043087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.043797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.045908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b830c36-c5e9-41d1-ac7c-aa1326635ae4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.076088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86cc4-df80-4526-b079-d2f4d265d8d8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.082274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.082957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.083243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.083568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.084306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:16.084665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.086345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.092350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 932/1871] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:15 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:16.093984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.094316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.094558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.100521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 940/1872] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:16.101671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.113443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf6b46-f23b-4f01-af1e-9a6d9b7fbb5f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.131304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] MFA Rules not processed for user `c16d64a16a094243abb17d781dbfef42`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:16.133197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:16.196940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:16.200362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.201104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.201504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.201860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.202631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.205296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.236758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 933/1873] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:16.237626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.240857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 941/1874] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:16.244652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.250740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c675-02e9-4594-aeec-047b2bd2a52c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.257735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.258387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cfab9-08c3-40a9-8d77-08cb79e6344f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.258887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.259128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.259416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.260112 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.262354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.269715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.270387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.270681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.270993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.271904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.273517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.286143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 942/1875] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:16.287502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.289837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:16.294218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 934/1876] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:16.295162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.351023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ff1f-5691-4aee-9f85-8a7b558360b4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.365539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f944ba-df74-476d-a477-f63ca3ec8fce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.375464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.376651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.376947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.377237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.378094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.380625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:16.381107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.394328 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 935/1877] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:16.395104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.405731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a1c5-434d-4875-baf2-1c148af1326e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.409744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.410398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.410508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.410677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.410975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.411185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.411476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.411756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.412443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.414387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.436472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 936/1878] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:16.437496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.447585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98167479-ad5a-40bd-b09d-bac2605ee4d9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.452483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.453469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.453751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.454031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] PATH_INFO: `/v3/projects/85a0599b290e4639bca77576af2f47a0/users/d2e645497755434dabaa63201bedc581/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.469655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a0599b290e4639bca77576af2f47a0, user_id=d2e645497755434dabaa63201bedc581, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.470793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.472241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.472423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b0dd7-a3a4-4a84-a97d-831055240eb9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.481797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 943/1879] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:16.482960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.486826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 937/1880] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:16 2022] PUT /identity/v3/projects/85a0599b290e4639bca77576af2f47a0/users/d2e645497755434dabaa63201bedc581/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:16.497036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd646-4863-426a-b33e-def2d025d486 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.497154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.497450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.497736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.525978 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] MFA Rules not processed for user `d2e645497755434dabaa63201bedc581`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:16.527240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:16.561792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.562466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.562746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.564362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.565128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.566724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.577327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 944/1881] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:16.581849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.593793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19011c94-00b9-424f-8c08-b802479bb1b8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.597930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 938/1882] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:16.599852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.600519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.600807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.601155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.601819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.602365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.603477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.612983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e1d21-c7d0-407a-a5bf-2f6246b064ef None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.614805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 945/1883] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:16.618836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.621858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.622734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.622863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.623124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.623964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.627824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.629650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4df19b-faff-4955-a807-d4320e4f1d7c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.630992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:16.635971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.637031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.637849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.637849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] PATH_INFO: `/v3/projects/c8099cc88ca540369d6795c2a97db757/users/9885b1ea3cd6405bab7baa37c81e26d4/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.660420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8099cc88ca540369d6795c2a97db757, user_id=9885b1ea3cd6405bab7baa37c81e26d4, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.661629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 939/1884] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:16.663288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad06040-f3fb-4388-8cea-b83408af8a83 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.665546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.685900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 946/1885] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:16 2022] PUT /identity/v3/projects/c8099cc88ca540369d6795c2a97db757/users/9885b1ea3cd6405bab7baa37c81e26d4/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:16.692166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.692466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.692792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.720640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] MFA Rules not processed for user `9885b1ea3cd6405bab7baa37c81e26d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:16.721861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:16.744176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71eb61-2c95-4c9b-8eb4-69f1050fd7b4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.763887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:16.834721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 947/1886] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:16.839683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.851967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.852762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.853111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.853522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.854235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.856482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.889868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 940/1887] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:16.893707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.905137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda63459-09db-4728-8e4f-b16bb052c18e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.910908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.911594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.911877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.912160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.912965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.914544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.929556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 941/1888] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:16.930481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:16.942987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41264a3-f139-4345-afaa-3f89c7a34a46 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:16.948996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:16.949957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:16.950248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:16.951009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] PATH_INFO: `/v3/projects/8b13a872776f43b6ad6a0c114ef17942/users/0c10b961eb9843f7a5898ca79bded7f3/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:16.969558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b13a872776f43b6ad6a0c114ef17942, user_id=0c10b961eb9843f7a5898ca79bded7f3, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:16.972313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:16.997699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 942/1889] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:16 2022] PUT /identity/v3/projects/8b13a872776f43b6ad6a0c114ef17942/users/0c10b961eb9843f7a5898ca79bded7f3/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:17.003232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.013361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19201d44-b049-4d2c-a01b-794f46ab7e39 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.106811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.107020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.107291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.107576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.108274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.110209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.127003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:17.146871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398bfdc-b17b-4987-a10f-c7d0c99e3224 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.170296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.171075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.171363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.172289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.175247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 948/1890] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:16 2022] POST /identity/v3/projects => generated 385 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:17.178220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.178980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.181396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2ec77-0655-4f47-bcf7-79e4ef555a70 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.247178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 943/1891] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:17.251471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865545c2-dfc6-4c8f-ba04-1c6e45cf29f1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.262412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865545c2-dfc6-4c8f-ba04-1c6e45cf29f1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.262412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865545c2-dfc6-4c8f-ba04-1c6e45cf29f1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.262412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 944/1892] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:09:17 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:09:17.267581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865545c2-dfc6-4c8f-ba04-1c6e45cf29f1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.325561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf4877-54c3-4f7b-9649-9fb708507cab admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.339976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.340662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.340951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.341231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.341933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.343936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d2fcb-73e1-44f0-a225-ff60b74813f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.352831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865545c2-dfc6-4c8f-ba04-1c6e45cf29f1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.358522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:17.366876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.367948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.368307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.368686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] PATH_INFO: `/v3/projects/8b13a872776f43b6ad6a0c114ef17942/users/0c10b961eb9843f7a5898ca79bded7f3/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.372432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b13a872776f43b6ad6a0c114ef17942, user_id=0c10b961eb9843f7a5898ca79bded7f3, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.376209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.382262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 949/1893] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:17 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:17.387270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.388235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.388695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.402248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 945/1894] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:17 2022] PUT /identity/v3/projects/8b13a872776f43b6ad6a0c114ef17942/users/0c10b961eb9843f7a5898ca79bded7f3/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:17.403240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.438289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:17.443317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:17.553971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27161101-17a3-4443-9b03-ad36a79c22d6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.563725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.564468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.564758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.565041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.565790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.567463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0057bf1-e7b4-4b4b-b716-4771b6ab4730 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.580351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 946/1895] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:17.581865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.582156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.582430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.597617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 950/1896] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:17.598924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.613164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d89bba-fdef-4df4-a898-b1751997dec8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.613699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] MFA Rules not processed for user `0c10b961eb9843f7a5898ca79bded7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:17.615455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:17.619036 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.620339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.620628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.621028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] PATH_INFO: `/v3/projects/ca811f6f88634731bb2174e13bf25099/users/17a499234ec545b3b048655871cdecf8/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.644665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca811f6f88634731bb2174e13bf25099, user_id=17a499234ec545b3b048655871cdecf8, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.648050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.674685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 951/1897] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:17 2022] PUT /identity/v3/projects/ca811f6f88634731bb2174e13bf25099/users/17a499234ec545b3b048655871cdecf8/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:17.679381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.707464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:17.716332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 947/1898] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:17.731464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.733114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:17.774928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245bb41-2a0f-4f32-acc3-45fba8b9fd27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.784431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.785553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.785830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.786107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.786870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.789163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.806432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 952/1899] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:17.806624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b9af7-6253-4231-b9fd-afefa0116d59 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.807744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:17.820493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2643642-ba6a-4ce6-b41d-47eacc81b678 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:17.833937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.835449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.835730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.836613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] PATH_INFO: `/v3/projects/ca811f6f88634731bb2174e13bf25099/users/17a499234ec545b3b048655871cdecf8/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.839896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca811f6f88634731bb2174e13bf25099, user_id=17a499234ec545b3b048655871cdecf8, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.842941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65412a-3eaa-4903-b207-a2e4c4300dfa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:17.860006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 953/1900] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:17 2022] PUT /identity/v3/projects/ca811f6f88634731bb2174e13bf25099/users/17a499234ec545b3b048655871cdecf8/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:17.870535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.871011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.871274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.872685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:17.873756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:17.874059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:17.874547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:17.937054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] MFA Rules not processed for user `17a499234ec545b3b048655871cdecf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:17.938576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:17.984541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:17.988345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:18.001790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 948/1901] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:09:18.003551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:18.005614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:18.016215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:18.102820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 954/1902] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:18.104161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:18.117201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:18.127095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b72c78-fb13-4c10-9a2f-5d0f3175fd5a service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:18.197827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6247d7a-2175-4e3d-9177-5405a2ebffce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:18.202336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:18.203113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:18.203496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:18.204974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:18.235551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:18.236485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:18.236772 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:18.237053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:18.248736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:18.250329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:18.262854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 949/1903] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:18.263881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:18.275844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b294cf8-ebf9-4901-a0fb-522d4d4c25e4 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:18.281516 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:18.283068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:18.283483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:18.283855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:18.298696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:18.303249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:18.315065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 955/1904] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:18.361254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:18.362698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658e031-3376-4c8e-90b8-d2691aac4379 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:18.372834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 950/1905] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:21.698234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:21.761561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f3f5f-af5d-4459-bb80-84eec73fe8a8 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:21.766075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:21.766997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:21.767280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:21.767582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] PATH_INFO: `/v3/users/d2e645497755434dabaa63201bedc581` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:21.768766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e645497755434dabaa63201bedc581)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:21.770674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:21.797512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:21.816797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e645497755434dabaa63201bedc581'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:21.823851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e645497755434dabaa63201bedc581'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:21.824481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e645497755434dabaa63201bedc581'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:21.836559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e645497755434dabaa63201bedc581'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:21.837192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e645497755434dabaa63201bedc581'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:21.849803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 956/1906] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:21 2022] DELETE /identity/v3/users/d2e645497755434dabaa63201bedc581 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:21.854297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:21.854747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:21.854886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:21.872311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:21.873654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:21.932634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 951/1907] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:21.942870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:21.944406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:22.004603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53e3be-0664-439c-9719-e52295fe61e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:22.006080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:22.006080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:22.006080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:22.006080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:22.016238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:22.018209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:22.027788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 957/1908] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:22.375719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:22.377738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:22.393021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6229928-bb08-4027-b5d9-122b1f4c9eed None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:22.398205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:22.399104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:22.399390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:22.399670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] PATH_INFO: `/v3/projects/85a0599b290e4639bca77576af2f47a0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:22.401022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a0599b290e4639bca77576af2f47a0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:22.402959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:22.425455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:22.449129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a0599b290e4639bca77576af2f47a0', 'request_id': 'req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.456805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a0599b290e4639bca77576af2f47a0', 'request_id': 'req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.463307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a0599b290e4639bca77576af2f47a0', 'request_id': 'req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.473411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] The token cache is being invalidate because project 85a0599b290e4639bca77576af2f47a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:22.473700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.474334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.475904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 952/1909] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:22 2022] DELETE /identity/v3/projects/85a0599b290e4639bca77576af2f47a0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:22.479829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:22.489389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b01e3-2fc1-4134-b260-a4c148822788 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:22.545010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:22.545912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:22.546191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:22.546464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] PATH_INFO: `/v3/users/0c10b961eb9843f7a5898ca79bded7f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:22.547676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c10b961eb9843f7a5898ca79bded7f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:22.549596 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:22.597774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c10b961eb9843f7a5898ca79bded7f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.599483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:22.606241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c10b961eb9843f7a5898ca79bded7f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.607231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c10b961eb9843f7a5898ca79bded7f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.620372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c10b961eb9843f7a5898ca79bded7f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.621040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-76846d50-3afa-4ab2-856b-f7e04998b512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c10b961eb9843f7a5898ca79bded7f3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:22.636330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 958/1910] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:22 2022] DELETE /identity/v3/users/0c10b961eb9843f7a5898ca79bded7f3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:23.090541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.092224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:23.115264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6dee8-ca33-4bce-b62c-c7ae64be7df5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.177551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.178418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.178703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.179015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] PATH_INFO: `/v3/projects/8b13a872776f43b6ad6a0c114ef17942` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.180207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b13a872776f43b6ad6a0c114ef17942)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.182108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.228967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b13a872776f43b6ad6a0c114ef17942', 'request_id': 'req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:23.237274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b13a872776f43b6ad6a0c114ef17942', 'request_id': 'req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:23.243420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b13a872776f43b6ad6a0c114ef17942', 'request_id': 'req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:23.253742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] The token cache is being invalidate because project 8b13a872776f43b6ad6a0c114ef17942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:23.253995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:23.254670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c662fbd-12b2-4bd6-b693-12e03c0e13fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:23.256367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 953/1911] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:23 2022] DELETE /identity/v3/projects/8b13a872776f43b6ad6a0c114ef17942 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:23.261326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.261771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.261882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.280677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:23.281897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:23.341580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 959/1912] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:23.349702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.350004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.350284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.368994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:23.370212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:23.378458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 954/1913] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:23.382711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.392407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b88a64-f814-4f80-9f6d-603885182b7b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.396980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.397640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.397929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.398209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.398971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:23.399283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.400892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.413714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 960/1914] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:23.418346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.437014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e9974-c490-46a5-8ca1-6b1da4b0da80 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.441724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.442376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.442665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.442984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.443679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.445580 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.472803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 955/1915] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:23.477728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.487704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cc601-cc99-473b-8d02-b69af3db70f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.543418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.544015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.544298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.544576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.545266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.547216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.567079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 961/1916] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:23.571152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.582160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba64ab2-ed32-4dce-8237-53e460488129 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.586723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.587495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.587671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.587945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.588698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.590333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.599230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:23.608853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 956/1917] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:23.610095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.620818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2487863-7277-44ff-86fb-73a0939d760e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.625662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.626327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.626647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.626966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.627813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.629422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cb780-f1ef-4b63-a727-1b2bb83f6305 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.641979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 962/1918] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:23.646754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:23.673493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ed71-cde7-4b52-8580-3bfd37dd829d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:23.679036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:23.679954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.680238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.680513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] PATH_INFO: `/v3/projects/8802b779151247b9a1a9f1bb1b54d53c/users/58c56c9f44b64d51872f9981e01de187/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.696721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8802b779151247b9a1a9f1bb1b54d53c, user_id=58c56c9f44b64d51872f9981e01de187, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:23.699426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:23.713661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 957/1919] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:23 2022] PUT /identity/v3/projects/8802b779151247b9a1a9f1bb1b54d53c/users/58c56c9f44b64d51872f9981e01de187/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:23.724939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:23.725305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:23.725650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:23.762720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] MFA Rules not processed for user `58c56c9f44b64d51872f9981e01de187`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:23.764981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:23.792747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:23.897169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 963/1920] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:23.901712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.033863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27b78d-5cb2-4858-ba23-d70a816ae9bd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.039686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.040364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.040654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.042366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.043132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.045256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.083307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.091040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 958/1921] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:23 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:24.112180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca964b9-d1d6-4a5e-a34b-986f3df7d984 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.206926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.207749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.208074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.208370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.209137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.211125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.294502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 964/1922] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:24 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:24.295716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.309043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ba9fb-058e-4765-803d-649e5f6e0d40 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.315087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.319436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.320005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.320494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.321447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.323974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.340813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 959/1923] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:24.345939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.369164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcde79-2aa4-45eb-af3c-eba198668bce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.374117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.375188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.375707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] PATH_INFO: `/v3/projects/8e437708f01147d09f36ab02ddfabf2a/users/6e4e89f3fbc24363bd6f077b69404956/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.409418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e437708f01147d09f36ab02ddfabf2a, user_id=6e4e89f3fbc24363bd6f077b69404956, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.411749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.445500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 965/1924] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:24 2022] PUT /identity/v3/projects/8e437708f01147d09f36ab02ddfabf2a/users/6e4e89f3fbc24363bd6f077b69404956/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:24.447898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.500917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b77d9-76e1-458b-9ee8-440a54a31b11 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.678091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.681060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.681356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.682827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.683982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.688190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6daf4f-6c81-47ff-b74e-bdf408faf25d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.715755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 960/1925] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:24.717610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:24.751782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f2518-c335-4f06-8ad6-b7f2a5f532b8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:24.756349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:24.757254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.757538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.757806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] PATH_INFO: `/v3/projects/8e437708f01147d09f36ab02ddfabf2a/users/6e4e89f3fbc24363bd6f077b69404956/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.765836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e437708f01147d09f36ab02ddfabf2a, user_id=6e4e89f3fbc24363bd6f077b69404956, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:24.769168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:24.805790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 966/1926] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:24 2022] PUT /identity/v3/projects/8e437708f01147d09f36ab02ddfabf2a/users/6e4e89f3fbc24363bd6f077b69404956/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:24.806635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:24.806961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:24.807282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:24.870402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] MFA Rules not processed for user `6e4e89f3fbc24363bd6f077b69404956`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:24.871786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:24.953774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:25.037662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 961/1927] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:25.041724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:25.044513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:25.149999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47031393-27cf-4655-a1f2-55abc78b39a8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:25.154514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:25.155206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:25.155624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:25.155950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:25.181902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:25.183765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909312f1-bad9-487f-9f27-333e5dd12c7f service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:25.202180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 967/1928] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:25.241524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:25.337049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ddcf3-039e-4f71-8ecf-15d7b2edc0c1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:25.341768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:25.342727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:25.343110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:25.343468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] PATH_INFO: `/v3/users/58c56c9f44b64d51872f9981e01de187` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:25.345291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c56c9f44b64d51872f9981e01de187)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:25.347343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:25.402593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c56c9f44b64d51872f9981e01de187'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:25.416582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c56c9f44b64d51872f9981e01de187'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:25.417485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c56c9f44b64d51872f9981e01de187'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:25.437975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c56c9f44b64d51872f9981e01de187'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:25.439029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c56c9f44b64d51872f9981e01de187'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:25.455980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 962/1929] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:25 2022] DELETE /identity/v3/users/58c56c9f44b64d51872f9981e01de187 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:25.460373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:25.460866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:25.460976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:25.479722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:25.480944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:25.490881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:25.557077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 968/1930] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:25.567011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:25.568522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:25.632373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c3b09-0c4f-469d-8968-0145e845f2e7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:25.637489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:25.638334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:25.638864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:25.639249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:25.646415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:25.651012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:25.653279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:25.675134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 963/1931] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:25.975378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:25.977353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:26.008610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e10f9c-53c7-4db6-a427-183924d21ee2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:26.020919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:26.021920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:26.022569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:26.023748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] PATH_INFO: `/v3/projects/8802b779151247b9a1a9f1bb1b54d53c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:26.027388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8802b779151247b9a1a9f1bb1b54d53c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:26.029527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:26.109155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8802b779151247b9a1a9f1bb1b54d53c', 'request_id': 'req-29d5cbd7-15b1-4132-8b7a-1e194bb66273'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.120236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8802b779151247b9a1a9f1bb1b54d53c', 'request_id': 'req-29d5cbd7-15b1-4132-8b7a-1e194bb66273'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.131954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8802b779151247b9a1a9f1bb1b54d53c', 'request_id': 'req-29d5cbd7-15b1-4132-8b7a-1e194bb66273'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.149180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] The token cache is being invalidate because project 8802b779151247b9a1a9f1bb1b54d53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:26.149473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.150298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.155132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 969/1932] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:25 2022] DELETE /identity/v3/projects/8802b779151247b9a1a9f1bb1b54d53c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:26.156177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:26.170063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeacef-1c35-4ee3-b14c-5c137289b807 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:26.254824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:26.256064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:26.256480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:26.256907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] PATH_INFO: `/v3/users/6e4e89f3fbc24363bd6f077b69404956` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:26.258281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4e89f3fbc24363bd6f077b69404956)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:26.260577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:26.314309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4e89f3fbc24363bd6f077b69404956'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.326052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4e89f3fbc24363bd6f077b69404956'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.336372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4e89f3fbc24363bd6f077b69404956'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.379683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4e89f3fbc24363bd6f077b69404956'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.380575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4e89f3fbc24363bd6f077b69404956'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:26.395595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 964/1933] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:26 2022] DELETE /identity/v3/users/6e4e89f3fbc24363bd6f077b69404956 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:27.073081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.075322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.089046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5cbd7-15b1-4132-8b7a-1e194bb66273 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.149630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.150941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.151298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.151585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] PATH_INFO: `/v3/projects/8e437708f01147d09f36ab02ddfabf2a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.152955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e437708f01147d09f36ab02ddfabf2a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.156149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.171735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.173874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.188807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3234-f7c4-4439-8c04-7ca564260077 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.194179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.194957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.195240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.195514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] PATH_INFO: `/v3/users/c16d64a16a094243abb17d781dbfef42` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.205571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16d64a16a094243abb17d781dbfef42)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.207965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.224725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:27.233808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e437708f01147d09f36ab02ddfabf2a', 'request_id': 'req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.244417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e437708f01147d09f36ab02ddfabf2a', 'request_id': 'req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.254850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e437708f01147d09f36ab02ddfabf2a', 'request_id': 'req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.273845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16d64a16a094243abb17d781dbfef42'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.278559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] The token cache is being invalidate because project 8e437708f01147d09f36ab02ddfabf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:27.278991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.279859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d5ba9516-3d7c-4ed0-909c-f7ea4847e72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.281489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16d64a16a094243abb17d781dbfef42'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.285439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 970/1934] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:27 2022] DELETE /identity/v3/projects/8e437708f01147d09f36ab02ddfabf2a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:27.285602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16d64a16a094243abb17d781dbfef42'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.286987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e0887-f9e5-4700-a6a8-3499996ce10b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.287305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e0887-f9e5-4700-a6a8-3499996ce10b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.287549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e0887-f9e5-4700-a6a8-3499996ce10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.295488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:27.298232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16d64a16a094243abb17d781dbfef42'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.299377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f92a154-4802-48d2-a091-c25bacfbee66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16d64a16a094243abb17d781dbfef42'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:27.311402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-509e0887-f9e5-4700-a6a8-3499996ce10b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:27.312745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-509e0887-f9e5-4700-a6a8-3499996ce10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.318021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 965/1935] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:27 2022] DELETE /identity/v3/users/c16d64a16a094243abb17d781dbfef42 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:27.318021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f2b9-8018-470e-ade1-cb2b51e3278e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.318021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f2b9-8018-470e-ade1-cb2b51e3278e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.318021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984f2b9-8018-470e-ade1-cb2b51e3278e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.319014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 966/1936] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:09:27.320251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09bd8-a480-45fc-88cf-daabaa916750 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.320535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09bd8-a480-45fc-88cf-daabaa916750 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.320811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09bd8-a480-45fc-88cf-daabaa916750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.340064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-56f09bd8-a480-45fc-88cf-daabaa916750 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:27.341391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-56f09bd8-a480-45fc-88cf-daabaa916750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.405560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 971/1937] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:27.409962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.413143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.413537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.439373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 967/1938] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:27.443028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.443028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.443028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.462358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:27.465839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:27.467091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.474291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.495686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 968/1939] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:27.496331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.499335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.603660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c5603-b1b7-400a-a074-c024d4c33590 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.604154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 972/1940] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:27.605273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.607729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.608383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.608670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.608949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.616721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12b609-f3df-4367-9b25-f294bd685c05 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.622040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.622716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.623735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.623735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.624223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:27.624537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.626379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.630624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.632603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.643062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 973/1941] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:27.644138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.646023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.650643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 969/1942] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:27.651679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.653183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.669048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329df1a-36e1-44e7-a7c8-87119ca8eb30 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.676225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.676879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.677164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.677444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.691307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.693630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.716656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 970/1943] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:27.717773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.730977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa15dd9-6c18-4e25-9fa7-939e63961378 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.737525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.738752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.739405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.739963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.742739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:27.746426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ecc40-9ef8-4630-813f-06108f91987a service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.751144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:27.756419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.757104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.757396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.757679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.758410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.760357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.826877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 971/1944] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:27.831182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:27.896699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.899083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:27.913254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 974/1945] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:27.921234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:27.941660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fd1f-df3c-46ec-9d01-913e679f5281 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:27.947236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:27.948026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:27.948365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:27.948901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:27.949841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:27.951967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.004778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 972/1946] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:27 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:28.009195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.028488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0358aad8-498b-4deb-86a5-ecfc36fc8cd2 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.038234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.039143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.039508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.039853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.040847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.042701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.072571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.073156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 975/1947] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:28.086618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b49a4-3593-446c-9271-67cab0b7fcc4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.093661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.095855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.096366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.096851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.097858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.099892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce6984-e2a4-43ba-b95d-bccd52a86e5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.121813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 973/1948] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:28.123143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.142443 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6b2e-de8b-41ed-9500-db7897cb8649 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.157668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.158630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.159784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.159784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] PATH_INFO: `/v3/projects/c3f826739edf4f66bfef4cb553b2f566/users/0b41cabe862840d0a5a6ced8d526f1e8/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.180380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f826739edf4f66bfef4cb553b2f566, user_id=0b41cabe862840d0a5a6ced8d526f1e8, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.184009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.224109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 976/1949] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:28 2022] PUT /identity/v3/projects/c3f826739edf4f66bfef4cb553b2f566/users/0b41cabe862840d0a5a6ced8d526f1e8/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:28.235760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.236340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.236976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.287027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] MFA Rules not processed for user `0b41cabe862840d0a5a6ced8d526f1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:28.291856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.310964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.313047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.439832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 974/1950] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:28.444148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.466832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959befbb-0287-4b9c-bde8-6d5791956fdf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.480376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.481267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.481550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.481824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] PATH_INFO: `/v3/users/04942747ea4e4ed4a17dce9b954caadc` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.484517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04942747ea4e4ed4a17dce9b954caadc)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.486795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.563484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f570d41-14d3-44b0-9370-920378711b23 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.584141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04942747ea4e4ed4a17dce9b954caadc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.585522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.587427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.588914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.589601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.591032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.593826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04942747ea4e4ed4a17dce9b954caadc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.595080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.595204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04942747ea4e4ed4a17dce9b954caadc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.613326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:28.615449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04942747ea4e4ed4a17dce9b954caadc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.617132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04942747ea4e4ed4a17dce9b954caadc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.635459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 977/1951] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:28 2022] DELETE /identity/v3/users/04942747ea4e4ed4a17dce9b954caadc => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:28.636530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.637890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.643086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 975/1952] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:28 2022] POST /identity/v3/projects => generated 393 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:28.646514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.648358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.713161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:28.729328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920ee90-5958-4519-9882-2a79faab148b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.734345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.735280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.735571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.735848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] PATH_INFO: `/v3/projects/3960585bb5274a34b138638da4f508a5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.737064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3960585bb5274a34b138638da4f508a5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.739021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.739438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8a84c-0eb9-41aa-9621-6513f22d8d2c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.745223 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.745998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.746432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.746797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.747596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.749505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.781389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 976/1953] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:28 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:28.785751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.796590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3960585bb5274a34b138638da4f508a5', 'request_id': 'req-642639a2-1954-4ae8-9598-437895990a83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.798864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5442c-feb6-47fd-aa68-adb0884efa21 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.804688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3960585bb5274a34b138638da4f508a5', 'request_id': 'req-642639a2-1954-4ae8-9598-437895990a83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.817799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3960585bb5274a34b138638da4f508a5', 'request_id': 'req-642639a2-1954-4ae8-9598-437895990a83'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.834556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] The token cache is being invalidate because project 3960585bb5274a34b138638da4f508a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:28.835280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.836056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-642639a2-1954-4ae8-9598-437895990a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:28.838124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 978/1954] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:28 2022] DELETE /identity/v3/projects/3960585bb5274a34b138638da4f508a5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:28.844012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.844466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.845360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.867531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:28.868568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.905534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.906348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.906728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.907181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.908005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.909694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.924902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 977/1955] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:28.924902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.926661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.941233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b0ac0-bc62-451c-bbb1-d0a515a7ddd7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.946182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.947135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.953507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.953620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 979/1956] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:28.953716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] PATH_INFO: `/v3/projects/c2afbdee29794a3285d6eaf1bdd3c3bb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.953833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:28.953833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:28.954677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2afbdee29794a3285d6eaf1bdd3c3bb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.956632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.963320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f4f2d-939c-44ad-9a1f-0d176aa333d0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:28.970719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:28.971633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:28.971910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:28.972186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] PATH_INFO: `/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f/users/df420d26bcdd423eab48424e962df39e/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:28.988002 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4556d78da04b70b3e21fa5b9ef950f, user_id=df420d26bcdd423eab48424e962df39e, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:28.990680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fcf35-297c-49ce-8fd2-546427da44f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:28.993161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:29.012321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 980/1957] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:28 2022] PUT /identity/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f/users/df420d26bcdd423eab48424e962df39e/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:29.013827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.014919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.015238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.029922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2afbdee29794a3285d6eaf1bdd3c3bb', 'request_id': 'req-cd5815d9-645a-4176-9ea5-7eb3a50597eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.036491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:29.037859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.040406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2afbdee29794a3285d6eaf1bdd3c3bb', 'request_id': 'req-cd5815d9-645a-4176-9ea5-7eb3a50597eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.054918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2afbdee29794a3285d6eaf1bdd3c3bb', 'request_id': 'req-cd5815d9-645a-4176-9ea5-7eb3a50597eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.076157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] The token cache is being invalidate because project c2afbdee29794a3285d6eaf1bdd3c3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:29.076739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.077553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.079790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 978/1958] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:28 2022] DELETE /identity/v3/projects/c2afbdee29794a3285d6eaf1bdd3c3bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:29.080388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.082046 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.122841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 981/1959] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:29.122841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.122841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.129349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef88a1-47a4-4982-8a97-f59f5293855c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.134339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.135264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.135544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.135817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] PATH_INFO: `/v3/users/bfb77a2726f542f88fa1f123c1b42257` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.137034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb77a2726f542f88fa1f123c1b42257)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.137560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:29.139374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.154700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5815d9-645a-4176-9ea5-7eb3a50597eb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.160615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.161281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.161563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.161839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.162571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.164253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.175443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 979/1960] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:29.176274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.188174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa8e6-09c0-4b5a-b48d-9782ed117c20 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.194435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.194435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.194837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.195020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.195755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:29.196069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.197674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.205827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb77a2726f542f88fa1f123c1b42257'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.212980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 980/1961] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:29.215883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.223851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb77a2726f542f88fa1f123c1b42257'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.224547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb77a2726f542f88fa1f123c1b42257'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.229159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1499f-f47c-4b0b-a63b-c0407ff799f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.234693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.236259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.236541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.236932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] PATH_INFO: `/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f/users/df420d26bcdd423eab48424e962df39e/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.239047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4556d78da04b70b3e21fa5b9ef950f, user_id=df420d26bcdd423eab48424e962df39e, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.239699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb77a2726f542f88fa1f123c1b42257'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.240607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a466a25b-c55e-49ce-9138-b09219c417e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb77a2726f542f88fa1f123c1b42257'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.241717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.253663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 981/1962] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:29 2022] PUT /identity/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f/users/df420d26bcdd423eab48424e962df39e/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:29.254283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.255833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 982/1963] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:29 2022] DELETE /identity/v3/users/bfb77a2726f542f88fa1f123c1b42257 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:29.265107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.269355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.269908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.270130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.307013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] MFA Rules not processed for user `df420d26bcdd423eab48424e962df39e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:29.308812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.381438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cc1d0-940e-4342-a052-6bd164b41d82 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.394378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:29.423898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 983/1964] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:29.436520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.439952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.466348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.468358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.468487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.468710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.469466 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.471519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.520851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 982/1965] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:29 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:29.527337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.548284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4041e2e-e0de-4647-bd26-0fa19f82474c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.618971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:29.645808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.646689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.647157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.647587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.651154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1e1cc-6429-4702-8fdc-3597d9db86f8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.666758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.667641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.667937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.668266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.669151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.671368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.726258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 983/1966] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:29 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:29.732059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.743543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe2f63-89d8-4161-b58d-fec96d6cb9a2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.748779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.749494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.749774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.750049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.751419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.753045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.780212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 984/1967] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:29.780212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.782633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.799353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ec02b-6948-471f-9191-a03d7a980f92 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.800041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.802130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.805127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.806346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.806795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.807236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] PATH_INFO: `/v3/projects/50eaa81ab24341e7ab8054f311c0a873` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.808640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] RBAC: Authorizing `identity:delete_project(project_id=50eaa81ab24341e7ab8054f311c0a873)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.811735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.821873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 984/1968] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:09:29.824094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.844087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cff062-7f39-4dbe-82f5-012250c5ac90 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.852311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.852996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.853444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.853728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:29.854457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:29.859814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:29.874876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 985/1969] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:29.879647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.898922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aea4b-5bf0-45b9-b5d4-8779d5c0ef94 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:29.913249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50eaa81ab24341e7ab8054f311c0a873', 'request_id': 'req-f485ee2b-981b-4025-8f7c-25e84ab8aaab'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.925953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50eaa81ab24341e7ab8054f311c0a873', 'request_id': 'req-f485ee2b-981b-4025-8f7c-25e84ab8aaab'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.934040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50eaa81ab24341e7ab8054f311c0a873', 'request_id': 'req-f485ee2b-981b-4025-8f7c-25e84ab8aaab'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.946451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] The token cache is being invalidate because project 50eaa81ab24341e7ab8054f311c0a873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:29.946867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.947625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:29.952458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 985/1970] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:29 2022] DELETE /identity/v3/projects/50eaa81ab24341e7ab8054f311c0a873 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:29.952458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:29.952458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:29.967913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:29.984360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:29.985962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:29.986191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:29.986455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] PATH_INFO: `/v3/projects/84e29c9ff0e74f14b70465eaa5e3eef9/users/b311abd603644594910dee5d3846c5d8/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.005430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e29c9ff0e74f14b70465eaa5e3eef9, user_id=b311abd603644594910dee5d3846c5d8, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.009142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549e33f-32fc-4108-8d41-6854efa4f3c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.016175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ee2b-981b-4025-8f7c-25e84ab8aaab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.020376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.021032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.021320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.021596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.023670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.031683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 986/1971] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:29 2022] PUT /identity/v3/projects/84e29c9ff0e74f14b70465eaa5e3eef9/users/b311abd603644594910dee5d3846c5d8/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:30.034025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94086-d2ff-47b8-944a-284afa62ebf5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.034630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94086-d2ff-47b8-944a-284afa62ebf5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.035178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc94086-d2ff-47b8-944a-284afa62ebf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.059744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0bc94086-d2ff-47b8-944a-284afa62ebf5 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:30.061217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bc94086-d2ff-47b8-944a-284afa62ebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.099669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.101136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f087-2d0c-4cfd-89e0-2ac2a573c2e3 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.112385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 986/1972] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:30.113688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.113989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.114302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.126460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:30.136909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 987/1973] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:30.146089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.146388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.146675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.146944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] MFA Rules not processed for user `b311abd603644594910dee5d3846c5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:30.149108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.168605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:30.169791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.179816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 988/1974] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:30.183571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.200313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31f384-1a88-41cb-8cc4-eaf9c08956bc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.206283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.206984 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.207436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.213395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.214141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:30.214420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.216355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.229950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 989/1975] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:09:30.234302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.240395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 987/1976] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:30.245038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.246860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.249330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba1c69-7677-42d1-965b-136c548d4a7c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.255547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.256208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.256487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.256759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.257536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.259607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.266168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b0ca1-015e-449f-9c5b-fb9333cfb39c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.273958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.275057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.275476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.275862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.277191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.280432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.294065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 990/1977] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:30.299124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.323646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 988/1978] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:30.325082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.401882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77e3a3-ed73-4afc-ac18-328467d5a232 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.443693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7709-093d-434a-b0a6-1dd1a4ca7e96 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.449822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:30.450175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.451038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.451433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.452012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.452522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.455000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.488241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 989/1979] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:30.491533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.500054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.501517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.501904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.502327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.503238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.506945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432624c-0f2b-4811-9233-f8185821ba58 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.509051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.523014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.524042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.524445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.524908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.525794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.530257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.545581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 990/1980] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:30.546008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 991/1981] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:30.547263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.549701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.561700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b5f96-c217-48a7-a5a5-a185036a728c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.566045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2798e69-5e11-4df4-b093-23ec477e46e9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.569196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.571174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.571535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.571886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.572142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.572919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.574065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.574542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.575233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] PATH_INFO: `/v3/projects/5c88140821c2467ba3f36bf37ee65fba/users/ba00f6c3cfe142ec82284e38020826ce/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.575729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.596750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 991/1982] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:30.597647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c88140821c2467ba3f36bf37ee65fba, user_id=ba00f6c3cfe142ec82284e38020826ce, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.598153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.601104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.611765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912387d-320a-497a-b1ab-89c753a0ba76 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.627496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 992/1983] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:30 2022] PUT /identity/v3/projects/5c88140821c2467ba3f36bf37ee65fba/users/ba00f6c3cfe142ec82284e38020826ce/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:30.632259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.681301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:30.728617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.730916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.731216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.731638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.732253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.735218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.765029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 992/1984] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:30.766200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.778384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab8f7-7b32-413d-bfb4-609797856029 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.787947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81ef1a-153f-42e5-a188-3b926eec54e9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.790556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.791780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.792057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.792335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.793084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.797319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.800476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.801439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.801726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.802003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] PATH_INFO: `/v3/projects/bd414fa553e740d0ac9ad39d72595c9e/users/9cb5cfb89ba5471298e859bc09a34d7a/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.824820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 993/1985] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:30.826980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:30.842486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd414fa553e740d0ac9ad39d72595c9e, user_id=9cb5cfb89ba5471298e859bc09a34d7a, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.846877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20081dcf-fb7b-463f-b943-859cb24a745b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.850722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52979-29f1-41cb-912d-d4702b7a2d55 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:30.859528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:30.860904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.861197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.861478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] PATH_INFO: `/v3/projects/5c88140821c2467ba3f36bf37ee65fba/users/ba00f6c3cfe142ec82284e38020826ce/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.863743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c88140821c2467ba3f36bf37ee65fba, user_id=ba00f6c3cfe142ec82284e38020826ce, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:30.866621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531d5a3-6355-417d-9679-fcf5cf5871c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:30.884203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 994/1986] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:30 2022] PUT /identity/v3/projects/5c88140821c2467ba3f36bf37ee65fba/users/ba00f6c3cfe142ec82284e38020826ce/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:30.889757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 993/1987] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:30 2022] PUT /identity/v3/projects/bd414fa553e740d0ac9ad39d72595c9e/users/9cb5cfb89ba5471298e859bc09a34d7a/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:30.906006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.906300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.906425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:30.906595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.906702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:30.907012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:30.962126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] MFA Rules not processed for user `ba00f6c3cfe142ec82284e38020826ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:30.963986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:30.966946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] MFA Rules not processed for user `9cb5cfb89ba5471298e859bc09a34d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:30.968237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:31.127690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 994/1988] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:31.135121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:31.161801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 995/1989] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:31.203418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:31.205368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:31.216749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:31.272369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b0dd8-507d-4498-a9c7-ccb3c20b61f6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:31.283605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:31.284258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:31.284541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:31.284814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:31.289157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f4e8f-e0c8-4480-b2a5-b4a9d1fdf3cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:31.298571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:31.299255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:31.299537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:31.300155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:31.303710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:31.305154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:31.331633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 995/1990] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:09:31.353462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:31.416449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:31.418685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f8e48-cba2-4870-9d70-e245e800153c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:31.430501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 996/1991] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:09:31.453580 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:31.477318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46ae4-9029-413e-8e74-d0435117104c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:31.484343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:31.485465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:31.485912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:31.486592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] PATH_INFO: `/v3/users/9cb5cfb89ba5471298e859bc09a34d7a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:31.488370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb5cfb89ba5471298e859bc09a34d7a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:31.491036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:31.552233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb5cfb89ba5471298e859bc09a34d7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:31.561644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb5cfb89ba5471298e859bc09a34d7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:31.562973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb5cfb89ba5471298e859bc09a34d7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:31.583938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb5cfb89ba5471298e859bc09a34d7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:31.585645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb5cfb89ba5471298e859bc09a34d7a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:31.606184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 996/1992] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:31 2022] DELETE /identity/v3/users/9cb5cfb89ba5471298e859bc09a34d7a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:31.612106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:31.612401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:31.612831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:31.642701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:31.644058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:31.786189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 997/1993] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:31.793587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:31.795333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:31.912375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e750-7eb7-4636-b8b4-b8d9afd8b4b5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:31.916774 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:31.917429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:31.917714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:31.917988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:31.935479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:31.938145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c31a64-0cf4-4f18-9d07-b0a04f5155f5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:31.962594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 997/1994] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:32.321188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:32.324915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:32.344270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33ef6-7e75-4189-8869-ef1a7e40e8c7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:32.353365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:32.354810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.356208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.356539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] PATH_INFO: `/v3/projects/bd414fa553e740d0ac9ad39d72595c9e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.358242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd414fa553e740d0ac9ad39d72595c9e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:32.363343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:32.450066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd414fa553e740d0ac9ad39d72595c9e', 'request_id': 'req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:32.457707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd414fa553e740d0ac9ad39d72595c9e', 'request_id': 'req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:32.462374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:32.468072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd414fa553e740d0ac9ad39d72595c9e', 'request_id': 'req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:32.505674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] The token cache is being invalidate because project bd414fa553e740d0ac9ad39d72595c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:32.506351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:32.507705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1d9cb5cf-eff0-4a28-ace4-e4740d8673be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:32.518799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 998/1995] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:32 2022] DELETE /identity/v3/projects/bd414fa553e740d0ac9ad39d72595c9e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:32.519676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.520121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.520568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.546877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:32.549145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:32.627975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 998/1996] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:32.636066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.636359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.636647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.657606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:32.659308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:32.670095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 999/1997] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:32.680922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:32.701458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504edb5-46f6-4199-a2b1-fe8b96f875f3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:32.708425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:32.709414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.709782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.710064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.712043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:32.712365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:32.715653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:32.719049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:32.741340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 999/1998] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:32.742025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:32.755188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37ace-e33a-4c6f-ac25-86e91b3d9087 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:32.766407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:32.767114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.767396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.767671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.768410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:32.770324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:32.800168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1000/1999] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:32 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:32.805365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:32.821606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107c397-8962-41f1-a8cc-1f71bc76ae5b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:32.916635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:32.917335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:32.917921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:32.918711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:32.919753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:32.922315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:32.970249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1000/2000] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:32 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:09:32.974305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:32.991687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48fb96-3dad-42ee-bd27-df11357ef2e2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.002365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.003170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.003467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.003727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.004469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.006140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.022119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1001/2001] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:33.022931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:33.045053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b359bb-a239-4d68-999b-a251726bc244 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.055742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.057009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.057284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.057568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.058962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.061246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa9fac-f55f-413a-b59d-d794734334e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.082816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1001/2002] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:33.085580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:33.098158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b408424-be26-44c4-99be-d88c3bee3065 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.103314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.105428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.105877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.107001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] PATH_INFO: `/v3/projects/2fb8d6dc5c3a4ccf8479b85aaed21bd7/users/a0cfe8fa5fe04a4c8147231cb0fb26fb/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.131070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb8d6dc5c3a4ccf8479b85aaed21bd7, user_id=a0cfe8fa5fe04a4c8147231cb0fb26fb, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.134812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.152680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1002/2003] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:33 2022] PUT /identity/v3/projects/2fb8d6dc5c3a4ccf8479b85aaed21bd7/users/a0cfe8fa5fe04a4c8147231cb0fb26fb/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:33.165199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.165495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.165777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.210864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] MFA Rules not processed for user `a0cfe8fa5fe04a4c8147231cb0fb26fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:33.210864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:33.284556 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1002/2004] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:33.295642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:33.297581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:33.354378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0273d5-670a-4ed7-86f9-0b821091800f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.358038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.358678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.358998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.359300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.367502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.368920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717b06f-13c1-4c5a-aae6-d6ebed7f7016 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.377915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1003/2005] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:33.441673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:33.443144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:33.550457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f7666-02c3-40c0-b64e-b4fd9fa66ec0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.555899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.556882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.557278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.557821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] PATH_INFO: `/v3/users/9885b1ea3cd6405bab7baa37c81e26d4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.559235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9885b1ea3cd6405bab7baa37c81e26d4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.561579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.570861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:33.632785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9885b1ea3cd6405bab7baa37c81e26d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:33.644537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9885b1ea3cd6405bab7baa37c81e26d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:33.645392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9885b1ea3cd6405bab7baa37c81e26d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:33.685431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9885b1ea3cd6405bab7baa37c81e26d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:33.686245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9885b1ea3cd6405bab7baa37c81e26d4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:33.712845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1003/2006] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:33 2022] DELETE /identity/v3/users/9885b1ea3cd6405bab7baa37c81e26d4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:33.716526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.716913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.717288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.743464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:33.744900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:33.832001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1004/2007] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:33.842031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:33.843888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:33.915843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73299d7-8a68-4adc-a6b2-088313df68f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:33.921953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:33.924385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:33.924738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:33.925082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:33.941497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:33.943427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:33.955346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1004/2008] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:34.091025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.093047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.105936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04377c0-1b61-4e02-a34d-1d2230871c9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.111674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.112696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.112986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.113267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] PATH_INFO: `/v3/users/0b41cabe862840d0a5a6ced8d526f1e8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.114563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b41cabe862840d0a5a6ced8d526f1e8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.116548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:34.165494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:34.217120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b41cabe862840d0a5a6ced8d526f1e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.240227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b41cabe862840d0a5a6ced8d526f1e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.240969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b41cabe862840d0a5a6ced8d526f1e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.259027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b41cabe862840d0a5a6ced8d526f1e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.259819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c35b91f-8a39-437d-b156-2ace3793c058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b41cabe862840d0a5a6ced8d526f1e8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.281138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.283770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.283958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1005/2009] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:34 2022] DELETE /identity/v3/users/0b41cabe862840d0a5a6ced8d526f1e8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:34.290163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.293976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.294658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.312566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c42d6-e0af-45bb-aff5-ebbd3185dad3 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.322384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:34.324785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.396185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1006/2010] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:34.397840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.403825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.416944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9a2b-40b5-43c8-bfb8-285ef140e0b8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.421092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.421593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.421957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.422240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.422512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.422690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] PATH_INFO: `/v3/users/b311abd603644594910dee5d3846c5d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.422830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.423083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] PATH_INFO: `/v3/projects/c8099cc88ca540369d6795c2a97db757` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.423767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b311abd603644594910dee5d3846c5d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.424282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8099cc88ca540369d6795c2a97db757)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.425663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:34.426175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:34.457647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:34.475882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8099cc88ca540369d6795c2a97db757', 'request_id': 'req-6603b9ac-21a9-4a5c-8393-1aafecd774ae'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.483207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b311abd603644594910dee5d3846c5d8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.483565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8099cc88ca540369d6795c2a97db757', 'request_id': 'req-6603b9ac-21a9-4a5c-8393-1aafecd774ae'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.492865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8099cc88ca540369d6795c2a97db757', 'request_id': 'req-6603b9ac-21a9-4a5c-8393-1aafecd774ae'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.495855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b311abd603644594910dee5d3846c5d8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.496650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b311abd603644594910dee5d3846c5d8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.506533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] The token cache is being invalidate because project c8099cc88ca540369d6795c2a97db757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:34.506709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.508616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.512108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1007/2011] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:34 2022] DELETE /identity/v3/projects/c8099cc88ca540369d6795c2a97db757 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:34.512108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.512352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b311abd603644594910dee5d3846c5d8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.512352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b311abd603644594910dee5d3846c5d8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.515010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.529016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1005/2012] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:34 2022] DELETE /identity/v3/users/b311abd603644594910dee5d3846c5d8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:34.532911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.534611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.553926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:34.594807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b9ac-21a9-4a5c-8393-1aafecd774ae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.626339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80538f7-e45d-4cde-a78f-1b687f3b4454 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.631730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.632620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.632898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.633197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] PATH_INFO: `/v3/users/17a499234ec545b3b048655871cdecf8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.634434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a499234ec545b3b048655871cdecf8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.636419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:34.651001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.651698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.651983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.652262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.654349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.668054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.669505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159405e-c86d-4a3c-816e-1176c1086ead service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:34.682448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1008/2013] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:34.684223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.684974 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.685405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.694405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a499234ec545b3b048655871cdecf8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.705748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a499234ec545b3b048655871cdecf8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.706541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a499234ec545b3b048655871cdecf8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.713873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:34.715581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.723702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a499234ec545b3b048655871cdecf8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.724864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a499234ec545b3b048655871cdecf8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:34.726007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1009/2014] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:34.739604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:34.742666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:34.743521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1006/2015] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:34 2022] DELETE /identity/v3/users/17a499234ec545b3b048655871cdecf8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:34.758877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc49aa-4772-4bd9-8c91-f8391c440cd7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:34.861196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:34.862017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:34.862305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:34.862583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:34.985640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:34.987953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.013957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1010/2016] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:35.303967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.305876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.325904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edb24c-2127-4fdb-8931-074ac863b0e1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:35.331855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:35.332890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.333885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.333885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] PATH_INFO: `/v3/projects/ca811f6f88634731bb2174e13bf25099` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.335185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca811f6f88634731bb2174e13bf25099)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:35.337329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.388936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:35.438985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.440502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.460301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45791169-9f70-44a0-be25-6df54a04733c service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:35.460822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca811f6f88634731bb2174e13bf25099', 'request_id': 'req-00e0c3d2-7da6-4962-a233-f746bfa9e680'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.471731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca811f6f88634731bb2174e13bf25099', 'request_id': 'req-00e0c3d2-7da6-4962-a233-f746bfa9e680'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.481371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca811f6f88634731bb2174e13bf25099', 'request_id': 'req-00e0c3d2-7da6-4962-a233-f746bfa9e680'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.497968 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] The token cache is being invalidate because project ca811f6f88634731bb2174e13bf25099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:35.498324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.499653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:35.500602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-00e0c3d2-7da6-4962-a233-f746bfa9e680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.504192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1007/2017] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:35 2022] DELETE /identity/v3/projects/ca811f6f88634731bb2174e13bf25099 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:35.512251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.512251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.521478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.569666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:35.576031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.583165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:35.584125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.584485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.584836 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] PATH_INFO: `/v3/projects/c3f826739edf4f66bfef4cb553b2f566` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.586588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f826739edf4f66bfef4cb553b2f566)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:35.591516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.591674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1008/2018] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:35.591674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.591674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.609159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e252c9-61d7-4765-a96a-bf282b30386f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:35.614441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:35.615427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.615778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.616053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] PATH_INFO: `/v3/projects/84e29c9ff0e74f14b70465eaa5e3eef9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.617642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e29c9ff0e74f14b70465eaa5e3eef9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:35.619697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.650111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f826739edf4f66bfef4cb553b2f566', 'request_id': 'req-839a2a53-b21c-45ab-bbc5-7365e667254b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.658296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f826739edf4f66bfef4cb553b2f566', 'request_id': 'req-839a2a53-b21c-45ab-bbc5-7365e667254b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.671849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f826739edf4f66bfef4cb553b2f566', 'request_id': 'req-839a2a53-b21c-45ab-bbc5-7365e667254b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.673665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e29c9ff0e74f14b70465eaa5e3eef9', 'request_id': 'req-f215449d-0510-437a-bea6-ba1bf32dff82'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.680529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e29c9ff0e74f14b70465eaa5e3eef9', 'request_id': 'req-f215449d-0510-437a-bea6-ba1bf32dff82'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.683494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] The token cache is being invalidate because project c3f826739edf4f66bfef4cb553b2f566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:35.683782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.684689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-839a2a53-b21c-45ab-bbc5-7365e667254b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.686238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1011/2019] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:35 2022] DELETE /identity/v3/projects/c3f826739edf4f66bfef4cb553b2f566 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:35.689397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e29c9ff0e74f14b70465eaa5e3eef9', 'request_id': 'req-f215449d-0510-437a-bea6-ba1bf32dff82'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.689515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.689515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.689515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.699636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] The token cache is being invalidate because project 84e29c9ff0e74f14b70465eaa5e3eef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:35.699920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.700602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.702309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1009/2020] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:35 2022] DELETE /identity/v3/projects/84e29c9ff0e74f14b70465eaa5e3eef9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:35.705796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.705796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.707489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:35.708667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.770589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1012/2021] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:35.770821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215449d-0510-437a-bea6-ba1bf32dff82 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:35.770935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.770935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.773083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:35.773952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.774247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.774509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] PATH_INFO: `/v3/users/df420d26bcdd423eab48424e962df39e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.775738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df420d26bcdd423eab48424e962df39e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:35.777673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.781394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeec424-7b5d-43e9-b03f-1617e19f17d7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:35.786618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:35.787522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:35.787810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:35.788089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] PATH_INFO: `/v3/users/ba00f6c3cfe142ec82284e38020826ce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:35.789295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba00f6c3cfe142ec82284e38020826ce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:35.791256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:35.806453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:35.829913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df420d26bcdd423eab48424e962df39e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.839052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df420d26bcdd423eab48424e962df39e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.839769 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df420d26bcdd423eab48424e962df39e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.842672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba00f6c3cfe142ec82284e38020826ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.845238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:35.851289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba00f6c3cfe142ec82284e38020826ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.852120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba00f6c3cfe142ec82284e38020826ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.852356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df420d26bcdd423eab48424e962df39e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.853018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df420d26bcdd423eab48424e962df39e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.865370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba00f6c3cfe142ec82284e38020826ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.865755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1010/2022] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:35 2022] DELETE /identity/v3/users/df420d26bcdd423eab48424e962df39e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:35.866083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba00f6c3cfe142ec82284e38020826ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:35.866617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:35.867995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:35.881155 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1013/2023] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:35 2022] DELETE /identity/v3/users/ba00f6c3cfe142ec82284e38020826ce => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:35.932373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6d9c-2eae-46f7-a827-d6633dd85ab7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.022059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.022739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.023075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.023357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.024102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:36.024413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.026032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.040443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1011/2024] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:36.045208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.046876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.069391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130cef3-5229-4591-9fb5-0cc80affc318 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.077315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.078006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.078310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.078893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.079870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.079961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.080563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.082553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.092860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f6457-f1f0-412d-8483-4e76ed1b86eb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.099437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.100662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.101055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.101348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] PATH_INFO: `/v3/users/a0cfe8fa5fe04a4c8147231cb0fb26fb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.105366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0cfe8fa5fe04a4c8147231cb0fb26fb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.108281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.123798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1014/2025] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:36.124368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.179406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:36.193086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0cfe8fa5fe04a4c8147231cb0fb26fb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.211530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87020230-9b8e-4ac9-aa7c-0260f56a5f39 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.217265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.218300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.218649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.219860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.220585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.220992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0cfe8fa5fe04a4c8147231cb0fb26fb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.222977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.224165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0cfe8fa5fe04a4c8147231cb0fb26fb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.246001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0cfe8fa5fe04a4c8147231cb0fb26fb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.247283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde49041-f62e-479c-a24a-b0cad292c1c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0cfe8fa5fe04a4c8147231cb0fb26fb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.251855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1015/2026] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:36.256520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.259178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.268807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1012/2027] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:36 2022] DELETE /identity/v3/users/a0cfe8fa5fe04a4c8147231cb0fb26fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:36.270698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.271227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.271711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.276896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec61e38-0f48-43cc-9fc7-b551e442f864 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.292475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:36.294045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.350745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.351451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.351741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.352275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.353043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.354658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.366842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1016/2028] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:36.369497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1013/2029] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:36.371627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.386122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.388800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409697d9-b1f6-413b-93ef-1e55319f2cda admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.389998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.395636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.396449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.396848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.397196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.397921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.399837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.417776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1017/2030] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:36.418632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.423410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.452047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c875b6-8cb2-47b3-a6f8-55d6be4e180f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.467358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.468834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.468834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.469051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] PATH_INFO: `/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.481164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4556d78da04b70b3e21fa5b9ef950f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.489119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.506275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65ca7-34f4-41c9-a3bd-752884e82479 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.521756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.522429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.523314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.523314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.536719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:36.547970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.549480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.563976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1014/2031] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:36.565013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.566324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.580402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a63d42-07c6-46d6-903d-88f2003e9376 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.590060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4556d78da04b70b3e21fa5b9ef950f', 'request_id': 'req-7b871739-caee-4b51-9a58-bf886e226c89'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.597661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4556d78da04b70b3e21fa5b9ef950f', 'request_id': 'req-7b871739-caee-4b51-9a58-bf886e226c89'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.608259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4556d78da04b70b3e21fa5b9ef950f', 'request_id': 'req-7b871739-caee-4b51-9a58-bf886e226c89'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.621594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] The token cache is being invalidate because project bf4556d78da04b70b3e21fa5b9ef950f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:36.621933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.622612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.624680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1018/2032] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:36 2022] DELETE /identity/v3/projects/bf4556d78da04b70b3e21fa5b9ef950f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:36.625683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:36.630659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.651329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.652898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.653345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.653792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] PATH_INFO: `/v3/projects/5c88140821c2467ba3f36bf37ee65fba` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.656262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c88140821c2467ba3f36bf37ee65fba)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.659292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.661194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:36.726934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871739-caee-4b51-9a58-bf886e226c89 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:36.732916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:36.734130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.734952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.735993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] PATH_INFO: `/v3/projects/34ec042870bb4aacbfe3bb4d799702c1/users/15fe42beea8446a09fd2326e53e9c715/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.746061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c88140821c2467ba3f36bf37ee65fba', 'request_id': 'req-3c7b0852-e739-4da6-8e23-52f82520474c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.763241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c88140821c2467ba3f36bf37ee65fba', 'request_id': 'req-3c7b0852-e739-4da6-8e23-52f82520474c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.763582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ec042870bb4aacbfe3bb4d799702c1, user_id=15fe42beea8446a09fd2326e53e9c715, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:36.766714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de4681-2c98-4588-9285-cbcefe074037 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:36.775886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c88140821c2467ba3f36bf37ee65fba', 'request_id': 'req-3c7b0852-e739-4da6-8e23-52f82520474c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.788545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1019/2033] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:36 2022] PUT /identity/v3/projects/34ec042870bb4aacbfe3bb4d799702c1/users/15fe42beea8446a09fd2326e53e9c715/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:36.790057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805fba-ad56-4c9e-bc2a-e7bb6796a6ee None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.790354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805fba-ad56-4c9e-bc2a-e7bb6796a6ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.790463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] The token cache is being invalidate because project 5c88140821c2467ba3f36bf37ee65fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:36.790636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805fba-ad56-4c9e-bc2a-e7bb6796a6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.790864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.791752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c7b0852-e739-4da6-8e23-52f82520474c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:36.793702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1015/2034] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:36 2022] DELETE /identity/v3/projects/5c88140821c2467ba3f36bf37ee65fba => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:36.799912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7152f-20ee-4100-80bd-eaf6435e6579 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.799912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7152f-20ee-4100-80bd-eaf6435e6579 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.799912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7152f-20ee-4100-80bd-eaf6435e6579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.811745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa805fba-ad56-4c9e-bc2a-e7bb6796a6ee None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:36.813594 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa805fba-ad56-4c9e-bc2a-e7bb6796a6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.827485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-77a7152f-20ee-4100-80bd-eaf6435e6579 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:36.832291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-77a7152f-20ee-4100-80bd-eaf6435e6579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:36.947589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1020/2035] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:36.947589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.947589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.947589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:36.977641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1016/2036] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:36.982849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29a681-f9ff-4153-b51e-020b0887b577 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:36.982849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29a681-f9ff-4153-b51e-020b0887b577 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:36.982849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29a681-f9ff-4153-b51e-020b0887b577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.013189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] MFA Rules not processed for user `15fe42beea8446a09fd2326e53e9c715`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.014658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.034083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7a29a681-f9ff-4153-b51e-020b0887b577 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.039765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:37.040331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a29a681-f9ff-4153-b51e-020b0887b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.061914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1017/2037] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.061914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.061914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.061914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.083893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.085236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.105088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1018/2038] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.106137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.117179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee7f7e-e007-466e-91fa-a0be9b8b5e06 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.122751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.123558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.123913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.124177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.124921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:37.125229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.126937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.140356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1019/2039] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:37.141858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.153859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1021/2040] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.155491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eabfc8-e935-4a42-97e3-add9e4e08aee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.155604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.163973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.164276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.164625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.165062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.165805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:37.166176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.168084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.179060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd7c7-e4a4-4b2a-981c-0e1fd2c08b4f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.185089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.185772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.197753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.198012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.201356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1020/2041] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:37.201356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.201652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.201734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.203258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.232613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a34550-841d-4e2f-a315-35ab3e1e3d00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.238156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.239070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.239356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.239636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] PATH_INFO: `/v3/projects/2fb8d6dc5c3a4ccf8479b85aaed21bd7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.240822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb8d6dc5c3a4ccf8479b85aaed21bd7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.241935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1022/2042] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:37.242751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.242964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.247007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.293103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:37.295925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb8d6dc5c3a4ccf8479b85aaed21bd7', 'request_id': 'req-4bf0aef5-f4d3-48b6-a62c-5299a011512f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.304805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb8d6dc5c3a4ccf8479b85aaed21bd7', 'request_id': 'req-4bf0aef5-f4d3-48b6-a62c-5299a011512f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.312291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb8d6dc5c3a4ccf8479b85aaed21bd7', 'request_id': 'req-4bf0aef5-f4d3-48b6-a62c-5299a011512f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.314977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292ad32-1d58-42cb-b3f5-d4c8c0eccc47 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.323010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] The token cache is being invalidate because project 2fb8d6dc5c3a4ccf8479b85aaed21bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:37.323303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.324009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.328828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1021/2043] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:37 2022] DELETE /identity/v3/projects/2fb8d6dc5c3a4ccf8479b85aaed21bd7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:37.328828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.328828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.368866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.369516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.369808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.370085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.371986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.389931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0aef5-f4d3-48b6-a62c-5299a011512f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.394680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.395368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.395661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.396031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.396806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.398966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.429064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1022/2044] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:37.429064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.430778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.433044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.434971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef6f21-aebd-43e1-a42e-dd85e8359718 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.444600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1023/2045] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:37.446039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.446334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.446614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.453667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:37.470593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.471843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.499249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c32fa7-0913-480a-bd78-c44631e00974 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.504034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.504590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.504876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.505157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.505856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.507813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.528438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1023/2046] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:37.532316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.532778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1024/2047] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.533672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.534987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.543028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0db20c-20f5-4d52-9a20-3af9f5e78d8c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.548195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.549058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.549285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.549564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.550254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.552191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.553795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b550f-26c0-4cd8-98a7-754de4e31ddd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.558821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.560171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.560946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.561428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] PATH_INFO: `/v3/users/15fe42beea8446a09fd2326e53e9c715` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.562857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15fe42beea8446a09fd2326e53e9c715)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.566025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.574392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1024/2048] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:37.575857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.587216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c5a3-2925-4b74-9391-d59f996db7df admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.592353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.593012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.593295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.593571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.594830 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.596499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1dcc6-e1d2-4090-a27f-4b58ae7139c8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.608753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1025/2049] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:37.610484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.610796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.611080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.624238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15fe42beea8446a09fd2326e53e9c715'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.628721 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.630092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.631922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15fe42beea8446a09fd2326e53e9c715'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.632638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15fe42beea8446a09fd2326e53e9c715'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.645218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15fe42beea8446a09fd2326e53e9c715'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.645932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15fe42beea8446a09fd2326e53e9c715'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:37.647065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1026/2050] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.648028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.651842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.665299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f0bdc-0ea8-41de-9086-7476b1d563c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.666071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1025/2051] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:37 2022] DELETE /identity/v3/users/15fe42beea8446a09fd2326e53e9c715 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:37.666993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.668301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.675405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=4,keystone:SELECT=88,keystone:INSERT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:37.726662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.727355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.727643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.727948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.728648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.730258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.733703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21201efd-e650-459c-988c-3320187cb86d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.738349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.739044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.739326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.739602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.742506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1027/2052] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:37.742650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.742784 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.742912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6292640-5bfa-40a7-a7d7-c462043dfe0b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.752686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e4920-bcde-4948-b78d-9f9d8f67897e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.753165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1026/2053] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:37.754374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.754662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.755029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.759424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.760067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.760421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.760628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.761350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:37.761658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.763322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.772250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:37.773457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.777332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1028/2054] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:37.778134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.784322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1027/2055] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:37.785367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.793595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7681e00-360f-47f3-bb2e-6c570f6884a0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.797456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081fe5-fc24-4d93-bc22-15badc8d1f61 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.798727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.799542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.799892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.800244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.801210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.803896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.804020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.804020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.804215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.804583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] PATH_INFO: `/v3/projects/06cbe20c96124114b507db0f228197a5/users/4ba146ac792644b299ac9a4d0432f37c/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.807216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:37.815247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1029/2056] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:37.815247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.827098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cbe20c96124114b507db0f228197a5, user_id=4ba146ac792644b299ac9a4d0432f37c, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.829029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683d42e-c351-406e-b694-f8e47e74a1f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.831403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.834061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.834958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.835314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.835852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.836454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.838949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.847825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1028/2057] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:37 2022] PUT /identity/v3/projects/06cbe20c96124114b507db0f228197a5/users/4ba146ac792644b299ac9a4d0432f37c/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:37.848966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.858711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:37.877590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1030/2058] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:37.878497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:37.926841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac454-6fd6-470d-ac75-26ddb0ebd137 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.949150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665096-84ad-44d8-9f47-02f58b4f036c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:37.954479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.955515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.955860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.956193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] PATH_INFO: `/v3/projects/52d2124fdaa64965a6e9a6a7834a28d8/users/7e65476ce6994ec09cbab99fb5834b19/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.979974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d2124fdaa64965a6e9a6a7834a28d8, user_id=7e65476ce6994ec09cbab99fb5834b19, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:37.982606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6023-19f1-46a7-bcf8-52adf31ae047 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:37.991389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:37.992161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.992604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:37.992937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:37.997547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1031/2059] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:37 2022] PUT /identity/v3/projects/52d2124fdaa64965a6e9a6a7834a28d8/users/7e65476ce6994ec09cbab99fb5834b19/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:37.999563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ab15-3d25-461e-9c06-8ddb402bc8c7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:37.999563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ab15-3d25-461e-9c06-8ddb402bc8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.000495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8ab15-3d25-461e-9c06-8ddb402bc8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.016620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.018194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.030860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1029/2060] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:38.032345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.034440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0cf8ab15-3d25-461e-9c06-8ddb402bc8c7 None None] MFA Rules not processed for user `4ba146ac792644b299ac9a4d0432f37c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:38.035827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0cf8ab15-3d25-461e-9c06-8ddb402bc8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.043689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb319b8a-f017-48de-aaf4-63e93cce9aad service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.079945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:38.117194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.117890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.118175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.118454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.119238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.121807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.125215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1032/2061] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:38.126681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de048e-28dd-4956-a358-166ffadd823e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.127016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de048e-28dd-4956-a358-166ffadd823e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.127279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de048e-28dd-4956-a358-166ffadd823e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.159035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1030/2062] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:38.160466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.162536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.167271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-92de048e-28dd-4956-a358-166ffadd823e None None] MFA Rules not processed for user `7e65476ce6994ec09cbab99fb5834b19`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:38.168806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-92de048e-28dd-4956-a358-166ffadd823e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.210845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:38.248785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9decc0-1ca8-4788-824a-25a5710851f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.255264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.255966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.256298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.256516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.267280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.269226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.279966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1031/2063] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:38.282332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.296399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a65fa-3069-4686-8827-d1757cabdac1 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.298963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1033/2064] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:38.303593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.304271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.304601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.304760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de048e-28dd-4956-a358-166ffadd823e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.306078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.307032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.309722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.317906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de048e-28dd-4956-a358-166ffadd823e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.323376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.324269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.324623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.324964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.325405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1032/2065] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:38.325822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.328000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.330476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.348512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87eb762-e7d1-440d-822a-bc14db447d98 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.356986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.358477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.358763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.359068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.360248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.361462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.371505 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1034/2066] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:38.374829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1033/2067] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:38.377850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.378053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.394119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda99ad-96b1-4664-b483-900e58ba919f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.459959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f0e99-f485-4cbd-a3dc-7c8cb633f9e8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.467926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.468607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.468888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.469189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.469941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.476420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.488168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.489903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.490416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.490922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] PATH_INFO: `/v3/projects/c6dbb0de53f84649b7c1a19b9df0a2e2/users/ec188cc29d45428688e6d2df3b2cbc5e/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.513829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dbb0de53f84649b7c1a19b9df0a2e2, user_id=ec188cc29d45428688e6d2df3b2cbc5e, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.516502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1407619-8a72-4c71-9f5a-1b6ee7c5f331 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.520792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1035/2068] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:38.524965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.541524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1034/2069] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:38 2022] PUT /identity/v3/projects/c6dbb0de53f84649b7c1a19b9df0a2e2/users/ec188cc29d45428688e6d2df3b2cbc5e/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:38.548736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25843de-d77d-40ac-902f-338ac434c39b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.555772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.556282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.557105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.585126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:38.618216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] MFA Rules not processed for user `ec188cc29d45428688e6d2df3b2cbc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:38.619740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.634715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.635675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.636100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.636513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.637474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.639667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.668599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1036/2070] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:09:38.676600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.679810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.709177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01032-6c6d-4448-a38f-0da26b1d0d5a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.722870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.723137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.723417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.723691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] PATH_INFO: `/v3/projects/34ec042870bb4aacbfe3bb4d799702c1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.724904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ec042870bb4aacbfe3bb4d799702c1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.726813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.764784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1035/2071] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:38.765440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.787922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d757daf-e986-4429-bfd8-a1bead119a75 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.798595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ec042870bb4aacbfe3bb4d799702c1', 'request_id': 'req-21f24035-103a-40fe-974f-3bc0a24d649f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:38.811335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ec042870bb4aacbfe3bb4d799702c1', 'request_id': 'req-21f24035-103a-40fe-974f-3bc0a24d649f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:38.821781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ec042870bb4aacbfe3bb4d799702c1', 'request_id': 'req-21f24035-103a-40fe-974f-3bc0a24d649f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:38.824240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:38.835456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] The token cache is being invalidate because project 34ec042870bb4aacbfe3bb4d799702c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:38.835865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:38.836947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:38.838602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1037/2072] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:38 2022] DELETE /identity/v3/projects/34ec042870bb4aacbfe3bb4d799702c1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:38.842807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.845171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.871496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.872166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.872448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.872732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.873458 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.875103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9a79-11d9-458a-b396-af226b5b1fc0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.885774 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1036/2073] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:38.890467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.890467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.890709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.908534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f24035-103a-40fe-974f-3bc0a24d649f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.909796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:38.911029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.913711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.914468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.914881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.915261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.917151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.919603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1037/2074] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:38.920455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:38.926187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:38.939204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33278-f348-4326-83ca-5e33df1573d9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:38.944119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:38.945014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.945296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.945571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] PATH_INFO: `/v3/projects/c714961333914b62b309a6fe0518159b/users/367b17d99cce4982a10307558493d4be/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.963033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c714961333914b62b309a6fe0518159b, user_id=367b17d99cce4982a10307558493d4be, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.966316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868995e-e459-456b-8760-72bdf66163ed admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:38.983572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1038/2075] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:38 2022] PUT /identity/v3/projects/c714961333914b62b309a6fe0518159b/users/367b17d99cce4982a10307558493d4be/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:38.985161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:38.985444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:38.985715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:38.995627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:38.997324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21801bef-7d5b-42f2-8724-ac1e24621d54 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.009552 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1038/2076] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:39.011890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.012201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.012456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.023509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:39.025931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:39.028760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:39.045075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] MFA Rules not processed for user `367b17d99cce4982a10307558493d4be`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:39.046356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:39.115001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1039/2077] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:39.119188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.129640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693f54-ece2-45d5-88c6-93a25e987d3e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.135115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.135783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.136065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.136340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.137066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:39.137388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.139105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1039/2078] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:39.139231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.142864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.153797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1040/2079] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:39.157623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169444e-b7aa-4e72-9bb0-e54b0dac8533 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.158310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.164086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.164846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.165127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.165445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.166686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.168816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.170822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2d2bd-20ae-4fe4-8f18-69524a010584 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.177688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.178374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.178681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.178965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.179657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.181611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.217789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1040/2080] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:39.222114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.237615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1041/2081] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:39.241569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.332284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a503b-25ad-4440-8235-ea377549a5a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.362406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:39.368465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd7d5a-6745-4ec9-848c-8551dc1ff8fd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.376402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.377576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.377835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.378648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.387343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.387343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.416870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1042/2082] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:39.419691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.435343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606430a-0b4b-471d-b373-9f6d852cb170 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.454216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.454916 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.455200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.455474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.456219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.457206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.457812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.458130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.458511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.458915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.459717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.462000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.474392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1043/2083] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:39.475815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.488247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bb44-39c6-46af-a918-b87be61917ad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.492907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1041/2084] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/users => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:39.497297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.497987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.498273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.502176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.503003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.504662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.509592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.522621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1044/2085] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:39.526501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.541035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40ef8a-ee61-4808-8f5b-565956690799 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.550868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909fcb4-9ae7-4140-b1bd-9a167b094067 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.551001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.551001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.551001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.551248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.551519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.551671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] PATH_INFO: `/v3/projects/30efec78809b4a9f962f6b866e41d5f5/users/c49c807477084e33a713cc42c9be05e4/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.551806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.552200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.552995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.555069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.562603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:39.571270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30efec78809b4a9f962f6b866e41d5f5, user_id=c49c807477084e33a713cc42c9be05e4, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.571980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1042/2086] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:39.576740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.577928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404ab26-7a6e-4285-bbc1-a6e4c87c7b35 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.591304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada38f49-6031-4397-a219-3f421d8ac778 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.598294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1045/2087] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:39 2022] PUT /identity/v3/projects/30efec78809b4a9f962f6b866e41d5f5/users/c49c807477084e33a713cc42c9be05e4/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:39.610467 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.610777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.611394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.639136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] MFA Rules not processed for user `c49c807477084e33a713cc42c9be05e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:39.640403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:39.666790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.667866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.668251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.668635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] PATH_INFO: `/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce/users/ed931754eefe42b5a3d3d4a339bc2051/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.686098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a9f883c40947c5ac0e904ce7fef8ce, user_id=ed931754eefe42b5a3d3d4a339bc2051, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.689727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.718962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1043/2088] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:39 2022] PUT /identity/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce/users/ed931754eefe42b5a3d3d4a339bc2051/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:39.723258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.726703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1046/2089] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:39.735536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.737851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:39.840598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94c23-5043-47df-ad17-4d69bc5fd7a6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.842488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a9384-be1e-47a4-9a22-e9baf4c58910 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.848155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.848836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.849120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.849399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.850149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.851800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.867294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1044/2090] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:39.869278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:39.881180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741644d-199d-4360-b9f6-bbea70c0c22d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:39.887627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.890425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.890719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.892452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] PATH_INFO: `/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce/users/ed931754eefe42b5a3d3d4a339bc2051/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.896040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:39.897113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a9f883c40947c5ac0e904ce7fef8ce, user_id=ed931754eefe42b5a3d3d4a339bc2051, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:39.903684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c931c-23e9-41b9-8f19-defc9ac39c74 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:39.916217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:39.917550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.918080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.918554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:39.930841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1045/2091] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:39 2022] PUT /identity/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce/users/ed931754eefe42b5a3d3d4a339bc2051/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:39.937208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:39.937751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:39.938066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:40.001110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] MFA Rules not processed for user `ed931754eefe42b5a3d3d4a339bc2051`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:40.002438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:40.006024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:40.012815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:40.026634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1047/2092] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:40.100905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:40.145555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1046/2093] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:40.184624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:40.196624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f7081-dfe8-42c8-aeb8-9fe46d2a0276 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:40.278194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:40.278808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:40.279091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:40.279365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:40.281205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:40.352818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:40.354341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:40.367197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1048/2094] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:43.111742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:43.121336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cc8aa-985c-42cf-82ec-470e75f6f236 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:43.127093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:43.127711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:43.127985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:43.128253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:43.183639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:43.185313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:43.194188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1047/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:45.586369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:45.594762 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2990-b72f-4ac1-a2aa-1eaf1e9cb485 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:45.598408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:45.599083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:45.599368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:45.599646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:45.608534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:45.609993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad7d30-cf73-4f05-b94a-202a3c477869 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:45.619396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1049/2096] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:53.190158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=32 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:53.563262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:53.564794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:53.630325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1b008-5db2-40fa-8647-d82a451d0188 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:53.634866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:53.635725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:53.636003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:53.636274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] PATH_INFO: `/v3/users/4ba146ac792644b299ac9a4d0432f37c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:53.637440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba146ac792644b299ac9a4d0432f37c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:53.639387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:53.683740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba146ac792644b299ac9a4d0432f37c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:53.691105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba146ac792644b299ac9a4d0432f37c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:53.691709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba146ac792644b299ac9a4d0432f37c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:53.703931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba146ac792644b299ac9a4d0432f37c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:53.704508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba146ac792644b299ac9a4d0432f37c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:53.716510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1048/2097] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:09:53 2022] DELETE /identity/v3/users/4ba146ac792644b299ac9a4d0432f37c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:53.721782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:53.722268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:53.722379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:53.739773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:53.740978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:53.773120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:53.798073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1050/2098] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:53.807403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:53.808779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:53.864310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9588d-c7a5-4347-935a-4a1bb09cd1a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:53.867911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:53.868543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:53.868823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:53.869095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:53.878504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:53.879954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da542bb-41d5-4884-ba02-c5596456d461 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:53.889289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1049/2099] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:09:54.150682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.152130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:54.164183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb4e2d-de7c-47d6-b9ff-369940e1cda7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.169216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.170418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.170571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.171415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] PATH_INFO: `/v3/projects/06cbe20c96124114b507db0f228197a5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.173247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06cbe20c96124114b507db0f228197a5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.175837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.224752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06cbe20c96124114b507db0f228197a5', 'request_id': 'req-66f1aa83-b469-4a4a-91bb-1d05b76767e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:54.230948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06cbe20c96124114b507db0f228197a5', 'request_id': 'req-66f1aa83-b469-4a4a-91bb-1d05b76767e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:54.240361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06cbe20c96124114b507db0f228197a5', 'request_id': 'req-66f1aa83-b469-4a4a-91bb-1d05b76767e3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:54.253162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] The token cache is being invalidate because project 06cbe20c96124114b507db0f228197a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:09:54.253420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:54.254067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-66f1aa83-b469-4a4a-91bb-1d05b76767e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:09:54.256006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1051/2100] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:09:54 2022] DELETE /identity/v3/projects/06cbe20c96124114b507db0f228197a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:09:54.261480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.261993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.262154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.280396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:54.282367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:54.300579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:09:54.346676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1050/2101] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:54.354543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.354872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.355158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.373847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:54.375118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:54.386816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1052/2102] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:09:54.388159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.398520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f34691-ed3d-4d83-9e3a-b6b3a073f2ed None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.403160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.403814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.404099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.404381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.405114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:09:54.405427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.407654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.419074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1051/2103] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:09:54.423185 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.432249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5a6d-469c-4d17-9dde-c09c809f18d4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.437209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.437854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.438141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.438421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.439161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.441070 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.466071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1053/2104] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:09:54 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:54.470550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.480697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9e731-4239-484c-a1e5-db95fa95e6fe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.535425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.536014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.536295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.536570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.537246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.539181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.557458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1052/2105] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:09:54 2022] POST /identity/v3/users => generated 288 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:09:54.561730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.572515 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b5c6-7f3c-452e-b588-e41b7aeb3039 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.577113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.577748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.578635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.578954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.579673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.581255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.591428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1054/2106] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:54.595179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.604872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee1d9f-3fd5-4075-8b7b-d741fc79ff27 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.609632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.610281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.610590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.610862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.611667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.613340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021bb1b-ba91-455a-9836-eb4b027ffe59 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.623792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1053/2107] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:09:54.628392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.638542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c292d8-d0ad-46c4-801a-a41059790b64 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.643733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.644619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.644919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.645176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] PATH_INFO: `/v3/projects/b0142f982ea440219cc60a4ad4acdeeb/users/f82d5d236970471e8e5a35162fd31c61/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.659791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0142f982ea440219cc60a4ad4acdeeb, user_id=f82d5d236970471e8e5a35162fd31c61, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.662444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.676207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1055/2108] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:09:54 2022] PUT /identity/v3/projects/b0142f982ea440219cc60a4ad4acdeeb/users/f82d5d236970471e8e5a35162fd31c61/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:09:54.687309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.688118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.688403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.715518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] MFA Rules not processed for user `f82d5d236970471e8e5a35162fd31c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:09:54.716732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:54.781884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1054/2109] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:09:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:09:54.792196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:09:54.793955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:09:54.849855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251dcf-ada3-4088-b88f-8054f48cadc2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:09:54.853479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:09:54.854107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:09:54.854995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:09:54.855278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:09:54.863600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:09:54.865021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09f435-b5a3-4e6b-95b9-5e5597522a2b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:09:54.873790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1056/2110] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:09:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:04.778908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:04.869939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:14.014990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:14.016435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:14.088785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cde631-b24f-4f91-aab9-c44508e4d36f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:14.093376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:14.094284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:14.094626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:14.095037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] PATH_INFO: `/v3/users/7e65476ce6994ec09cbab99fb5834b19` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:14.096310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e65476ce6994ec09cbab99fb5834b19)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:14.098349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:14.150066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e65476ce6994ec09cbab99fb5834b19'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.156989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e65476ce6994ec09cbab99fb5834b19'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.157617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e65476ce6994ec09cbab99fb5834b19'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.169288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e65476ce6994ec09cbab99fb5834b19'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.170021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e65476ce6994ec09cbab99fb5834b19'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.182572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1055/2111] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:14 2022] DELETE /identity/v3/users/7e65476ce6994ec09cbab99fb5834b19 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:14.187849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:14.188330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:14.188432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:14.206300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:14.207618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:14.266588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1057/2112] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:14.276632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:14.277963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:14.334758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56c992-2537-4697-aa08-eb6cc70d0fbe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:14.338377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:14.339040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:14.339317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:14.339588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:14.348494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:14.349901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:14.359133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1056/2113] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:14.812802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:14.814692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:14.828492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e02a8f-fd20-4567-912a-2781c1316e2d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:14.834078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:14.835219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:14.835550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:14.835842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] PATH_INFO: `/v3/projects/52d2124fdaa64965a6e9a6a7834a28d8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:14.837215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d2124fdaa64965a6e9a6a7834a28d8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:14.839190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:14.890787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d2124fdaa64965a6e9a6a7834a28d8', 'request_id': 'req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.896853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d2124fdaa64965a6e9a6a7834a28d8', 'request_id': 'req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.904158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d2124fdaa64965a6e9a6a7834a28d8', 'request_id': 'req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.914581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] The token cache is being invalidate because project 52d2124fdaa64965a6e9a6a7834a28d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:14.914886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.915503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:14.917067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1058/2114] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:14 2022] DELETE /identity/v3/projects/52d2124fdaa64965a6e9a6a7834a28d8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:14.921193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:14.930604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7fc76-4ca6-4831-b1b1-3dddc50b3694 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:14.949299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:14.988158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:14.989014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:14.989289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:14.989558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] PATH_INFO: `/v3/users/367b17d99cce4982a10307558493d4be` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:14.990733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=367b17d99cce4982a10307558493d4be)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:14.992695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:15.040464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '367b17d99cce4982a10307558493d4be'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.049390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '367b17d99cce4982a10307558493d4be'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.050009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '367b17d99cce4982a10307558493d4be'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.063739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '367b17d99cce4982a10307558493d4be'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.064418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '367b17d99cce4982a10307558493d4be'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.077975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1057/2115] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:14 2022] DELETE /identity/v3/users/367b17d99cce4982a10307558493d4be => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:15.609611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:15.611097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:15.626687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61c80-e308-4ed1-9ce6-5ceffcb33fb9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:15.698975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:15.699914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:15.700262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:15.700616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] PATH_INFO: `/v3/projects/c714961333914b62b309a6fe0518159b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:15.702134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c714961333914b62b309a6fe0518159b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:15.704124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:15.744172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:15.778563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c714961333914b62b309a6fe0518159b', 'request_id': 'req-2527a6e4-9d98-4012-918e-32fc7372223b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.801166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c714961333914b62b309a6fe0518159b', 'request_id': 'req-2527a6e4-9d98-4012-918e-32fc7372223b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.822871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c714961333914b62b309a6fe0518159b', 'request_id': 'req-2527a6e4-9d98-4012-918e-32fc7372223b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.840195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] The token cache is being invalidate because project c714961333914b62b309a6fe0518159b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:15.840518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.843607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:15.845388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1059/2116] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:15 2022] DELETE /identity/v3/projects/c714961333914b62b309a6fe0518159b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:15.849537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:15.851227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:15.869996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d6594-790e-4451-8ad6-7c8bf042ef61 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.000668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.001718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.002296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.002898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] PATH_INFO: `/v3/users/ed931754eefe42b5a3d3d4a339bc2051` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.004332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed931754eefe42b5a3d3d4a339bc2051)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.006680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.070093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:16.074588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed931754eefe42b5a3d3d4a339bc2051'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.085136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed931754eefe42b5a3d3d4a339bc2051'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.086275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed931754eefe42b5a3d3d4a339bc2051'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.106156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed931754eefe42b5a3d3d4a339bc2051'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.107176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed931754eefe42b5a3d3d4a339bc2051'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.127012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1058/2117] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:15 2022] DELETE /identity/v3/users/ed931754eefe42b5a3d3d4a339bc2051 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:16.351291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.352803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.365776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527a6e4-9d98-4012-918e-32fc7372223b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.457651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.458172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.459005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.459290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.459694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.459983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] PATH_INFO: `/v3/users/ec188cc29d45428688e6d2df3b2cbc5e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.461810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec188cc29d45428688e6d2df3b2cbc5e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.464129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.480809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbdff5-c569-4431-b23e-8db0958b8e92 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.487371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.488590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.499711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.500008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] PATH_INFO: `/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.501432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a9f883c40947c5ac0e904ce7fef8ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.503509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.528746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec188cc29d45428688e6d2df3b2cbc5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.538502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec188cc29d45428688e6d2df3b2cbc5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.539266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec188cc29d45428688e6d2df3b2cbc5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.559176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec188cc29d45428688e6d2df3b2cbc5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.560373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10df05dd-35dc-4a71-8d05-6f2a923034fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec188cc29d45428688e6d2df3b2cbc5e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.572507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a9f883c40947c5ac0e904ce7fef8ce', 'request_id': 'req-070a6482-f7a3-47ac-baf0-4e69d0552096'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.579255 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1060/2118] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:16 2022] DELETE /identity/v3/users/ec188cc29d45428688e6d2df3b2cbc5e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:16.583928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.584336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.584663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.586487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a9f883c40947c5ac0e904ce7fef8ce', 'request_id': 'req-070a6482-f7a3-47ac-baf0-4e69d0552096'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.602020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a9f883c40947c5ac0e904ce7fef8ce', 'request_id': 'req-070a6482-f7a3-47ac-baf0-4e69d0552096'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.607323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:16.609719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.616480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] The token cache is being invalidate because project f1a9f883c40947c5ac0e904ce7fef8ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:16.616891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.618525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-070a6482-f7a3-47ac-baf0-4e69d0552096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:16.620931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1059/2119] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:16 2022] DELETE /identity/v3/projects/f1a9f883c40947c5ac0e904ce7fef8ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:16.626236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7b7fa-35bd-47c0-b74d-dead69133100 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.626513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7b7fa-35bd-47c0-b74d-dead69133100 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.627809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7b7fa-35bd-47c0-b74d-dead69133100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.653097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dac7b7fa-35bd-47c0-b74d-dead69133100 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:16.654485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dac7b7fa-35bd-47c0-b74d-dead69133100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.659250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:16.705946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1061/2120] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:16.721100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.723196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.753670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1060/2121] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:16.761265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.761874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.762364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.793964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:16.796152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:16.808738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1061/2122] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:16.810167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.820406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c540162-beeb-40b3-a81a-85e5790c002f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.825341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.826011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.826297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.826580 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.827976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:16.828296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.830182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.832716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36725439-0694-4c56-a8b1-ab7387ee98ad None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.836961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.837823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.838258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.838710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.844114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1062/2123] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:10:16.850006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.857925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.861571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.868319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3788-f8d7-4b8a-a3c4-62b5765f848d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:16.873565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1062/2124] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:16.890420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:16.891225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:16.891391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:16.891885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:16.892738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:16.894488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:16.952568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1063/2125] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:16 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:16.954415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:16.969158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f134b-7acb-4b7d-83fd-02979d5e3d09 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.059579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.060753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.061436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.062054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.063441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.066022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.108112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1063/2126] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:16 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:17.108768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.116053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:17.140909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b813f-eecb-4178-90e8-34f3bde2fa75 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.152320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.156274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.160029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.160610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.163475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.169656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.191719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1064/2127] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:10:17.200157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.250323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c9119-0a1c-4b71-ba2b-2191a2997d70 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.263471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.264332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.264607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.265542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.266284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.274115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.295559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1064/2128] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:17.298890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.310493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36dd17-0ae8-419c-99ad-1a7046c82533 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.317620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.319140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.319633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.320098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] PATH_INFO: `/v3/projects/2c9d3e291460490496762f44871efac4/users/81c86423d6a645fe8fe8062daed5fcdd/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.330485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.332637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:17.339517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9d3e291460490496762f44871efac4, user_id=81c86423d6a645fe8fe8062daed5fcdd, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.342295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dece54-eacb-4fc1-b8f8-b425be717d81 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.351379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbbdbd-7dac-4f10-b4b2-8b914727506f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.358654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.359945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.360418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.360790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] PATH_INFO: `/v3/projects/c6dbb0de53f84649b7c1a19b9df0a2e2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.362912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6dbb0de53f84649b7c1a19b9df0a2e2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.365537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.386857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1065/2129] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:17 2022] PUT /identity/v3/projects/2c9d3e291460490496762f44871efac4/users/81c86423d6a645fe8fe8062daed5fcdd/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:17.398453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.398743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.399052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.450227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6dbb0de53f84649b7c1a19b9df0a2e2', 'request_id': 'req-5f238ca3-2316-45a4-9b59-ec13312f3a6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:17.453077 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:17.455772 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] MFA Rules not processed for user `81c86423d6a645fe8fe8062daed5fcdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:17.457050 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:17.464626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6dbb0de53f84649b7c1a19b9df0a2e2', 'request_id': 'req-5f238ca3-2316-45a4-9b59-ec13312f3a6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:17.478675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6dbb0de53f84649b7c1a19b9df0a2e2', 'request_id': 'req-5f238ca3-2316-45a4-9b59-ec13312f3a6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:17.495512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] The token cache is being invalidate because project c6dbb0de53f84649b7c1a19b9df0a2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:17.496366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:17.497579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5f238ca3-2316-45a4-9b59-ec13312f3a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:17.500336 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1065/2130] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:17 2022] DELETE /identity/v3/projects/c6dbb0de53f84649b7c1a19b9df0a2e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:17.505662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b07-ad54-4bef-9cf7-3c43d3f111b8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.506069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b07-ad54-4bef-9cf7-3c43d3f111b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.506591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0b07-ad54-4bef-9cf7-3c43d3f111b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.539506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-567f0b07-ad54-4bef-9cf7-3c43d3f111b8 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:17.541351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-567f0b07-ad54-4bef-9cf7-3c43d3f111b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:17.574865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1066/2131] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:17.582171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.586648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:17.636065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1066/2132] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:17.638044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.638334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.638921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.662360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:17.665247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:17.677735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1067/2133] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:17.678884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66864d-5d32-482f-9c4a-477befc98de7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.682147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.691467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.692602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880853b8-c2d8-4b15-afa4-29aa23f0527d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.696225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.696560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.696845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.706899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.710348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.710348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.710348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.710348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:17.710348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.712130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.715957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.717571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.754749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1067/2135] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:17.755745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.756263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1068/2135] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:10:17.792008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619127e5-3317-41ad-bd59-9ac114a612ee service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:17.800727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:17.801497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:17.801654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:17.801913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:17.802606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:17.804540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:17.864124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1068/2136] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:17 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:17.867888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:17.884186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abdda2-a8ed-4b23-90af-dcf4a38d8bd8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:18.000714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:18.001820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.007032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.007650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.010941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:18.016951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:18.067329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1069/2137] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:17 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:18.071573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:18.091889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55a995-65d8-4c52-be1e-872103ecf932 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:18.102914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:18.103677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.104762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.105042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.105786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:18.109236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:18.123019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1069/2138] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:18.127525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:18.141475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041842d-c7fc-485f-a4ff-a877ce4ab7de admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:18.150921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:18.151865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.152154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.152434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.153171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:18.154802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405d015-0bc3-4080-965d-cfed695be28e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:18.168572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1070/2139] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:10:18.173650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:18.195096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10438025-7f20-42c6-a454-ff62a6d57cfc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:18.213725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:18.214872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.216749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.217042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] PATH_INFO: `/v3/projects/44d80f2a562c4adf9e3dd0f5dc0d0fde/users/0b69518c9792452bb59f3a534c396117/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.258763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d80f2a562c4adf9e3dd0f5dc0d0fde, user_id=0b69518c9792452bb59f3a534c396117, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:18.265278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:18.292485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1070/2140] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:18 2022] PUT /identity/v3/projects/44d80f2a562c4adf9e3dd0f5dc0d0fde/users/0b69518c9792452bb59f3a534c396117/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:18.308402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2eb0-2b34-47da-a4aa-4d49ce001e63 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.308723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2eb0-2b34-47da-a4aa-4d49ce001e63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.309157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da2eb0-2b34-47da-a4aa-4d49ce001e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.355405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-39da2eb0-2b34-47da-a4aa-4d49ce001e63 None None] MFA Rules not processed for user `0b69518c9792452bb59f3a534c396117`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:18.356780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39da2eb0-2b34-47da-a4aa-4d49ce001e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:18.400694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:18.472736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1071/2141] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:18.485579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:18.487507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:18.496359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:18.562836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a12d-5a9e-4355-9d5c-9875cd2819d5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:18.567909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:18.568984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:18.569410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:18.569831 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:18.579176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:18.580608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354f951-4634-4904-b8c8-606b39af51b7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:18.589654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1071/2142] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:19.711287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:19.711858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:19.712011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:19.713457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1072/2143] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:10:19.725975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:19.726435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:19.726655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:19.743759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] MFA Rules not processed for user `41b5461ab9d746b2b45cbc1960dececd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:19.744988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:19.773163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:19.779736 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:19.818914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1072/2144] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:10:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:19.830550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:19.832007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:19.845847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05060fb-c27f-4153-b847-96f6807217d7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:19.851080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:19.851863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:19.852156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:19.852535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:19.862149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:19.864073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:19.870640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2079231-988b-4d1e-8b8f-96d74507dc53 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:19.875304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:19.876172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:19.876450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:19.876717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] PATH_INFO: `/v3/users/ec800c1fb0e74f2caabab848db0432db` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:19.877915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec800c1fb0e74f2caabab848db0432db)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:19.878644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1073/2145] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:19.880060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:19.930811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec800c1fb0e74f2caabab848db0432db'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:19.939328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec800c1fb0e74f2caabab848db0432db'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:19.940025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec800c1fb0e74f2caabab848db0432db'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:19.953411 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec800c1fb0e74f2caabab848db0432db'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:19.954094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-efc154e8-c638-4719-86d9-fe541b8b3470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec800c1fb0e74f2caabab848db0432db'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:19.966933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1073/2146] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:19 2022] DELETE /identity/v3/users/ec800c1fb0e74f2caabab848db0432db => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:20.151226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.163485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb6317-51ad-4f0c-ab58-620323dc8e26 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.227905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.228792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.229077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.229358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] PATH_INFO: `/v3/projects/fa10fec302f0464ead56582b81409441` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.230550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa10fec302f0464ead56582b81409441)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.232487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.266011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:20.279194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa10fec302f0464ead56582b81409441', 'request_id': 'req-596d59fd-a177-4f01-b6d2-659785f92530'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:20.286100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa10fec302f0464ead56582b81409441', 'request_id': 'req-596d59fd-a177-4f01-b6d2-659785f92530'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:20.292130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa10fec302f0464ead56582b81409441', 'request_id': 'req-596d59fd-a177-4f01-b6d2-659785f92530'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:20.302136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] The token cache is being invalidate because project fa10fec302f0464ead56582b81409441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:20.302419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:20.303065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-596d59fd-a177-4f01-b6d2-659785f92530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:20.304599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1074/2147] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:20 2022] DELETE /identity/v3/projects/fa10fec302f0464ead56582b81409441 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:20.309902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.310196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.310478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.329303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:20.330511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:20.370469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:20.390274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1074/2148] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:20.398497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.398819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.399100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.417788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:20.419029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:20.427710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1075/2149] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:20.432074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.443081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ab396-9b71-45e4-8335-b2ff3c69fb9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.447809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.448466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.448786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.449024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.449747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:20.450052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.451731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.463196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1075/2150] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:10:20.467404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.476459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2e39b-eb0c-4056-8bdb-44a8bdab9914 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.481288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.481934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.482228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.482499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.483242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.485639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.514422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1076/2151] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:20 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:20.518531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.528803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b5155-7336-413f-8787-ba29e0236e3f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.587310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.587972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.588254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.588524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.589239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.591171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.611192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1076/2152] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:20 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:20.615376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.627310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6cd-e7f2-478f-945b-6741b5538fd2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.632436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.633274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.633559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.633836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.634619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.636519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.647159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1077/2153] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:20.651233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.661775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adc51e-94d6-419b-a8a4-bd027fbc1977 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.666405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.667086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.667370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.667648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.668379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.669966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749f11-a1c6-4f4a-bd4b-cee558c78023 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.680690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1077/2154] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:20.685243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.695713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e888b4b-01e7-479b-9c1d-9f618e005c22 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.700333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.701205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.701496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.701777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] PATH_INFO: `/v3/projects/d3212384428441a0a8d04ee17a1322a7/users/b767fcae6e0046439e33cc0ea14ad60e/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.716107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3212384428441a0a8d04ee17a1322a7, user_id=b767fcae6e0046439e33cc0ea14ad60e, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.718758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.733144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1078/2155] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:20 2022] PUT /identity/v3/projects/d3212384428441a0a8d04ee17a1322a7/users/b767fcae6e0046439e33cc0ea14ad60e/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:20.738745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7a4f-dac1-43de-a5ab-cc84b216e2e4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.739189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7a4f-dac1-43de-a5ab-cc84b216e2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.739352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7a4f-dac1-43de-a5ab-cc84b216e2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.758364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d44b7a4f-dac1-43de-a5ab-cc84b216e2e4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:20.759631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d44b7a4f-dac1-43de-a5ab-cc84b216e2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:20.818384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1078/2156] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:20.827789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:20.829478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:20.886243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1b2aa-da8e-4970-9278-525279123688 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:20.889860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:20.890494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:20.890851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:20.891082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:20.900902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:20.902251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:20.912507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1079/2157] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:10:21.755403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:21.755403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:21.755403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:21.790987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] MFA Rules not processed for user `b767fcae6e0046439e33cc0ea14ad60e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:21.792219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:21.839968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:21.864768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1079/2158] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:21.869063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:21.879627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd9d2-f633-4e5b-b673-bcc24e72849a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:21.884144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:21.884790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:21.885069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:21.885343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:21.886038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:21.887974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:21.916494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1080/2159] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:21 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:21.917720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:21.928900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a43ab-7a89-4a7e-8ae6-3c6efae5b63b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:21.985839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:21.986437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:21.986724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:21.987042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:21.987727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:21.989716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:22.010231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1080/2160] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:21 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:22.014062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:22.025278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158a44-3310-4503-93d9-f1929c67a0fe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:22.029938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:22.030598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:22.030920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:22.031203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:22.031928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:22.033523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:22.044874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1081/2161] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:22.049634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:22.059794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e3169-50bc-4ea5-b287-2da2f2d1abae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:22.064948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:22.065815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:22.066102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:22.066380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] PATH_INFO: `/v3/projects/33a973d2182345e1bdd0c24165ee53fc/users/3922db22335447bfb666c369b378e767/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:22.082024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a973d2182345e1bdd0c24165ee53fc, user_id=3922db22335447bfb666c369b378e767, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:22.084742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:22.097476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1081/2162] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:22 2022] PUT /identity/v3/projects/33a973d2182345e1bdd0c24165ee53fc/users/3922db22335447bfb666c369b378e767/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:22.101584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:22.112118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934568ee-6a6e-4c98-a307-b9875df0fa24 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:22.132172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:22.168923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:22.169598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:22.169879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:22.170157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:22.171755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:22.173053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c342e79-0e96-4658-829f-3614a6b0b08d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:22.185397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1082/2163] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:22.190342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:22.200638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253571-5d4c-4a75-8d59-b77fb4bb4fe1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:22.205323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:22.206220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:22.206641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:22.206800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] PATH_INFO: `/v3/projects/33a973d2182345e1bdd0c24165ee53fc/users/3922db22335447bfb666c369b378e767/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:22.209270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a973d2182345e1bdd0c24165ee53fc, user_id=3922db22335447bfb666c369b378e767, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:22.211975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:22.222997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1082/2164] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:22 2022] PUT /identity/v3/projects/33a973d2182345e1bdd0c24165ee53fc/users/3922db22335447bfb666c369b378e767/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:23.108116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:23.108735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:23.108735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:23.136091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] MFA Rules not processed for user `3922db22335447bfb666c369b378e767`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:23.137293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:23.216315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1083/2165] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:24.524106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:24.588815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006095d-5ff3-4ca0-b623-161212307c4c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:24.594092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:24.594981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:24.595264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:24.595538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] PATH_INFO: `/v3/users/b767fcae6e0046439e33cc0ea14ad60e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:24.596705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b767fcae6e0046439e33cc0ea14ad60e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:24.598630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:24.636058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:24.646038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b767fcae6e0046439e33cc0ea14ad60e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.652999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b767fcae6e0046439e33cc0ea14ad60e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.653599 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b767fcae6e0046439e33cc0ea14ad60e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.665791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b767fcae6e0046439e33cc0ea14ad60e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.666408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b767fcae6e0046439e33cc0ea14ad60e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.678884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1083/2166] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:24 2022] DELETE /identity/v3/users/b767fcae6e0046439e33cc0ea14ad60e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:24.859085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:24.871576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4179b3c1-bde7-497d-89ac-6aaa7e0f02ed None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:24.933068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:24.933923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:24.934454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:24.934743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] PATH_INFO: `/v3/projects/d3212384428441a0a8d04ee17a1322a7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:24.935961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3212384428441a0a8d04ee17a1322a7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:24.937975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:24.982141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:24.983414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3212384428441a0a8d04ee17a1322a7', 'request_id': 'req-08cf2a5f-8fad-411a-99b5-2213829f452e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.991101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3212384428441a0a8d04ee17a1322a7', 'request_id': 'req-08cf2a5f-8fad-411a-99b5-2213829f452e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:24.997196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3212384428441a0a8d04ee17a1322a7', 'request_id': 'req-08cf2a5f-8fad-411a-99b5-2213829f452e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.007257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] The token cache is being invalidate because project d3212384428441a0a8d04ee17a1322a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:25.007544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.008174 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.010012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1084/2167] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:24 2022] DELETE /identity/v3/projects/d3212384428441a0a8d04ee17a1322a7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:25.014102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.016100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:25.029054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693aa942-f324-43f1-88e3-395ae2c58d3a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.086989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.087867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.088153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.088431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] PATH_INFO: `/v3/users/3922db22335447bfb666c369b378e767` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.089629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3922db22335447bfb666c369b378e767)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.091575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.136281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3922db22335447bfb666c369b378e767'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.143893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3922db22335447bfb666c369b378e767'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.144572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3922db22335447bfb666c369b378e767'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.157331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3922db22335447bfb666c369b378e767'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.157967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f9416d7-fb89-4d51-870a-8ffd26a7d1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3922db22335447bfb666c369b378e767'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.169769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1084/2168] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:25 2022] DELETE /identity/v3/users/3922db22335447bfb666c369b378e767 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:25.340692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.342289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:25.358577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf2a5f-8fad-411a-99b5-2213829f452e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.415237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.416025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.416314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.416587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] PATH_INFO: `/v3/projects/33a973d2182345e1bdd0c24165ee53fc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.417785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a973d2182345e1bdd0c24165ee53fc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.419739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.465094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a973d2182345e1bdd0c24165ee53fc', 'request_id': 'req-164b7113-5ee3-4f3c-8946-08d43ae0e21f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.472004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a973d2182345e1bdd0c24165ee53fc', 'request_id': 'req-164b7113-5ee3-4f3c-8946-08d43ae0e21f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.478096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a973d2182345e1bdd0c24165ee53fc', 'request_id': 'req-164b7113-5ee3-4f3c-8946-08d43ae0e21f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.488321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] The token cache is being invalidate because project 33a973d2182345e1bdd0c24165ee53fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:25.488608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.489240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-164b7113-5ee3-4f3c-8946-08d43ae0e21f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:25.490741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1085/2169] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:25 2022] DELETE /identity/v3/projects/33a973d2182345e1bdd0c24165ee53fc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:25.495643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.495961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.496245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.514360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:25.516215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:25.545514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:25.578589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1085/2170] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:25.586033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.586322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.586628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.604950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:25.606147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:25.614054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1086/2171] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:25.618171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.628335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987d626-36e4-4288-aa1d-e3603a9c181f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.633018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.633692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.634057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.634342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.635106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:25.635420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.637027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.652940 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1086/2172] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:10:25.654001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.663381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612259b-dc53-40ee-a61b-551f754f12c9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.668116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.668706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.668989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.669269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.669965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.672383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.697561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1087/2173] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:25 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:25.701713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.711604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8452d-10f8-48e0-99c9-c40bc033d810 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.768431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.769087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.769367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.769637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.770329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.772259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.793782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1087/2174] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:25 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:25.795010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.806277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b5e0c-4dff-4b74-984c-6d7da34cb4c2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.812054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.812770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.813073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.813359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.814240 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.815782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.826175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1088/2175] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:25.830087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.839921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507a1-fc27-4c08-b710-92ca413f6df7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.844563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.845208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.845491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.847082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.847880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.849496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292aa9d-a746-4907-90c7-3e750da444ae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.860803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1088/2176] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:25.865024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:25.875993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31666-883f-4457-91e6-72bab1f76b37 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:25.883717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:25.893104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.893393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.893674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] PATH_INFO: `/v3/projects/563b299739474e9bb1e9b52ffb54f92d/users/40812967594143268681b6952c450768/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.909654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=563b299739474e9bb1e9b52ffb54f92d, user_id=40812967594143268681b6952c450768, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:25.912324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:25.925974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1089/2177] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:25 2022] PUT /identity/v3/projects/563b299739474e9bb1e9b52ffb54f92d/users/40812967594143268681b6952c450768/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:25.936206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:25.937117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:25.937394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:25.962977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] MFA Rules not processed for user `40812967594143268681b6952c450768`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:25.964171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:26.027123 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1089/2178] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:26.032115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.036521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:26.094476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34279-c2e6-43e4-80ef-4aed70ba8c7f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.099081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.099727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.100007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.100281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.101000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.102941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.127709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1090/2179] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:26 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:26.131657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.142705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa522-4058-4d83-b436-0a0e6f6ec38c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.171489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:26.202697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.203320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.203601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.203874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.204798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.206720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.225728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1090/2180] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:26 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:26.229703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.240688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386567c-b925-4da3-bfd0-a78b036b550e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.245390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.246041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.246329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.246608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.247378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.248973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.260502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1091/2181] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:10:26.264637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.274309 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d0ea6-595c-499c-95a0-44e933e9ffd8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.278874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.279738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.280018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.280802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] PATH_INFO: `/v3/projects/efdc9b3337e6413cadc9b696bbb848f1/users/f83bb0ac9f59427c94c4b03374ddd7e9/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.295208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdc9b3337e6413cadc9b696bbb848f1, user_id=f83bb0ac9f59427c94c4b03374ddd7e9, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.298484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.311128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1091/2182] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:26 2022] PUT /identity/v3/projects/efdc9b3337e6413cadc9b696bbb848f1/users/f83bb0ac9f59427c94c4b03374ddd7e9/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:26.314958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.324905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624783-a95a-41b6-95df-d400c131e2c2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.387442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.388111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.388396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.388675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.389402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.391045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdee89-c380-4635-b8ee-3377b05413da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.401253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1092/2183] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:26.405686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.415910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e5d08-c2a8-4326-b48b-10ed5add0e3b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.420471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.421902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.422181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.422454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] PATH_INFO: `/v3/projects/efdc9b3337e6413cadc9b696bbb848f1/users/f83bb0ac9f59427c94c4b03374ddd7e9/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.424471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdc9b3337e6413cadc9b696bbb848f1, user_id=f83bb0ac9f59427c94c4b03374ddd7e9, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.427125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.437463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1092/2184] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:26 2022] PUT /identity/v3/projects/efdc9b3337e6413cadc9b696bbb848f1/users/f83bb0ac9f59427c94c4b03374ddd7e9/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:26.448393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.448689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.449028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.479401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] MFA Rules not processed for user `f83bb0ac9f59427c94c4b03374ddd7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:26.480553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:26.541171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:26.547622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1093/2185] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:26.558580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.560593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:26.617209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce83f2-1650-4c03-945a-caf030ca48b6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.620872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.621512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.621802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.622079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.631207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.632657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.642486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1093/2186] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:26.727661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:26.729129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:26.741613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270bde-48cf-4398-81ef-45b345254c30 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:26.745242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:26.745882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:26.746188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:26.746463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:26.803639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:26.805073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:26.814113 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1094/2187] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:31.067973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:31.069134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.155501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec2b713-ff3d-4058-b913-9106fdc74bbe service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:31.160972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:31.162126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.162462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.162990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] PATH_INFO: `/v3/users/0b69518c9792452bb59f3a534c396117` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.164432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b69518c9792452bb59f3a534c396117)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:31.166732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:31.196177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:31.216214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b69518c9792452bb59f3a534c396117'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.225167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b69518c9792452bb59f3a534c396117'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.226139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b69518c9792452bb59f3a534c396117'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.241322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b69518c9792452bb59f3a534c396117'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.241873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d733812-3011-4bd7-86ad-09072b9c230d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b69518c9792452bb59f3a534c396117'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.246243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:31.247783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.254844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1094/2188] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:31 2022] DELETE /identity/v3/users/0b69518c9792452bb59f3a534c396117 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:31.259312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.259615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.259912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.262512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df15b1-ed1c-4a99-a54d-aca277f9992f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:31.278218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:31.279471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.319498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:31.320365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.320646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.320925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] PATH_INFO: `/v3/users/f82d5d236970471e8e5a35162fd31c61` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.322150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82d5d236970471e8e5a35162fd31c61)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:31.324365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:31.339578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1095/2189] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:31.351964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:31.353486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.380911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82d5d236970471e8e5a35162fd31c61'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.389669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82d5d236970471e8e5a35162fd31c61'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.390348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82d5d236970471e8e5a35162fd31c61'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.403994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82d5d236970471e8e5a35162fd31c61'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.405490 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0f0dec8-b148-49d1-8181-25be14bf53e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82d5d236970471e8e5a35162fd31c61'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.418026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be21d7-add4-440d-83fc-d1d74d2843fe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:31.420433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1095/2190] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:31 2022] DELETE /identity/v3/users/f82d5d236970471e8e5a35162fd31c61 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:31.424414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:31.425281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.425511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.425633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.425915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.426210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.426643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.428666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.445740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:31.447291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:31.447604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:31.449358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.458235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1096/2191] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:31.461665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:31.526456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1096/2192] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:31.536925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:31.538585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.551341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a173d-e58d-4a45-9596-33fb95637b28 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:31.605544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:31.612510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:31.613347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.613807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.614293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.624747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:31.626358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a6089-275e-439e-ad1c-b7bbdb077fe0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:31.638343 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1097/2193] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:31.883554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:31.885244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:31.901163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e06f44-8e96-4ad0-873a-f249d7ca1aa1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:31.905940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:31.907512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:31.907814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:31.908066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] PATH_INFO: `/v3/projects/44d80f2a562c4adf9e3dd0f5dc0d0fde` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:31.909295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d80f2a562c4adf9e3dd0f5dc0d0fde)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:31.911281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:31.966002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d80f2a562c4adf9e3dd0f5dc0d0fde', 'request_id': 'req-b51eeb48-668b-415e-b6e7-6a77795b67e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.974976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d80f2a562c4adf9e3dd0f5dc0d0fde', 'request_id': 'req-b51eeb48-668b-415e-b6e7-6a77795b67e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.982135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d80f2a562c4adf9e3dd0f5dc0d0fde', 'request_id': 'req-b51eeb48-668b-415e-b6e7-6a77795b67e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.992912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] The token cache is being invalidate because project 44d80f2a562c4adf9e3dd0f5dc0d0fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:31.993303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.994004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:31.995835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1097/2194] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:31 2022] DELETE /identity/v3/projects/44d80f2a562c4adf9e3dd0f5dc0d0fde => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:32.004027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176964eb-12ea-4f62-b0bb-91b5276ab4ae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.004027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176964eb-12ea-4f62-b0bb-91b5276ab4ae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.004027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176964eb-12ea-4f62-b0bb-91b5276ab4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.023040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-176964eb-12ea-4f62-b0bb-91b5276ab4ae None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:32.024256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-176964eb-12ea-4f62-b0bb-91b5276ab4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.040531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.042333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.095138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1098/2195] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:32.102833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.103132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.103414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.122020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:32.123265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.131991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51eeb48-668b-415e-b6e7-6a77795b67e2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.136692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.137364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1099/2196] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:32.137730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.138325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.138888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] PATH_INFO: `/v3/projects/b0142f982ea440219cc60a4ad4acdeeb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.140690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0142f982ea440219cc60a4ad4acdeeb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.142298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.146647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.154717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5addb4-2ede-4b1b-826f-0d51dc0bbb91 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.159904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.160573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.160858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.161136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.161875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:32.162190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.163839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.176964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1100/2197] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:10:32.182106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.214549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca741d10-0504-4bea-9289-6ce1b11563a2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.221589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:32.230062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0142f982ea440219cc60a4ad4acdeeb', 'request_id': 'req-aa776c1a-3cb4-41fd-a124-113cfb021fd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:32.244004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0142f982ea440219cc60a4ad4acdeeb', 'request_id': 'req-aa776c1a-3cb4-41fd-a124-113cfb021fd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:32.254479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0142f982ea440219cc60a4ad4acdeeb', 'request_id': 'req-aa776c1a-3cb4-41fd-a124-113cfb021fd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:32.265788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] The token cache is being invalidate because project b0142f982ea440219cc60a4ad4acdeeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:32.266623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:32.267334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:32.270471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1098/2198] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:32 2022] DELETE /identity/v3/projects/b0142f982ea440219cc60a4ad4acdeeb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:32.294011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.294975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.295198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.295486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.296366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.298434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.345094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1101/2199] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:32 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:32.345332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.345332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.358519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa776c1a-3cb4-41fd-a124-113cfb021fd4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.426092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.426825 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.427119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.427401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.428403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.431428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.458972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1099/2200] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:32 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:32.460334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.472317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f86b7-67eb-42b4-87be-f16e30d9e721 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.477154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.477966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.478283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.478558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.479485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.481170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.492366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1102/2201] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:32.496762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.507591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1262d-4b60-4d12-99e4-c330b2eb2ff2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.512354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.513012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.513295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.513623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.514362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.516031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bb17c-9671-45e9-88fc-07bc84868b32 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.526939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1100/2202] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:32.531028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.540997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6558c-ff79-4373-b541-9b059050aa08 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.545732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.546646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.546966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.547770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] PATH_INFO: `/v3/projects/d8def43018744d7b8d5a5f422e44bf89/users/0a36e02c916b494180e68d3f231ee714/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.563611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8def43018744d7b8d5a5f422e44bf89, user_id=0a36e02c916b494180e68d3f231ee714, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.566281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.580536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1103/2203] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:32 2022] PUT /identity/v3/projects/d8def43018744d7b8d5a5f422e44bf89/users/0a36e02c916b494180e68d3f231ee714/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:32.585655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f7cd-c107-448b-a0b4-f62ef4430cfc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.585951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f7cd-c107-448b-a0b4-f62ef4430cfc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.586236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923f7cd-c107-448b-a0b4-f62ef4430cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.607009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d923f7cd-c107-448b-a0b4-f62ef4430cfc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:32.607009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d923f7cd-c107-448b-a0b4-f62ef4430cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.665936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1101/2204] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:32.675996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:32.677711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:32.699997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:32.735373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a096b8c-6290-4d3c-b0a2-c5648d37bf05 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:32.739128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:32.739766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:32.740051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:32.740330 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:32.749307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:32.751340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:32.760726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1104/2205] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:37.055554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.056058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.056058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.090465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] MFA Rules not processed for user `0a36e02c916b494180e68d3f231ee714`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:37.091451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:37.132924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:37.164415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1102/2206] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:37.175269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.187245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5305b-b6fd-46a2-aa77-8ec962c82d9e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.199182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.200083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.200217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.200505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.201406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.203424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.247398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1105/2207] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:37 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:37.248365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.266412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d511132-9299-4e0a-a191-4753c3081837 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.446720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.453462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.453752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.454032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.454754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.456736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.505146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1103/2208] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:37 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:37.508014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.524518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824281f5-47a0-4390-8d09-5e2b02a9d2cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.530523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.532161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.532416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.532695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.533792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.536676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.554188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1106/2209] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:37.558465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.571776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949bc22-722d-4d75-a46a-06f142ac0cf6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.581158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.582076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.583032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.583032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] PATH_INFO: `/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e/users/af8f9521b12d464cbd49422a77842b20/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.610038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d82e6fabd448d99fa6e8d56c17cf0e, user_id=af8f9521b12d464cbd49422a77842b20, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.612983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.630233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1104/2210] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:37 2022] PUT /identity/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e/users/af8f9521b12d464cbd49422a77842b20/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:10:37.634550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.646757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c64f4-6578-4646-8005-8f0c14cdb77d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.741718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.742386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.742666 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.742986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.743710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.745506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.756119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1107/2211] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:37.760005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:37.771194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f30d5-3301-481b-bc31-f1d7f265d360 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:37.776058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:37.776961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:37.777251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:37.777529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] PATH_INFO: `/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e/users/af8f9521b12d464cbd49422a77842b20/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:37.779521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d82e6fabd448d99fa6e8d56c17cf0e, user_id=af8f9521b12d464cbd49422a77842b20, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:37.782761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0d9b6-af90-49b5-b939-0558d23c83db admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:37.793964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1105/2212] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:37 2022] PUT /identity/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e/users/af8f9521b12d464cbd49422a77842b20/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:38.035937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:38.037784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:38.052242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664fca0-21cc-4732-b600-1782bfa60098 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:38.137299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:38.137924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:38.138220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:38.138503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] PATH_INFO: `/v3/users/c49c807477084e33a713cc42c9be05e4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:38.139947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49c807477084e33a713cc42c9be05e4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:38.143138 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:38.206378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:38.240402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49c807477084e33a713cc42c9be05e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:38.262429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49c807477084e33a713cc42c9be05e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:38.263231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49c807477084e33a713cc42c9be05e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:38.284065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49c807477084e33a713cc42c9be05e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:38.285224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49c807477084e33a713cc42c9be05e4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:38.314326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1108/2213] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:10:38 2022] DELETE /identity/v3/users/c49c807477084e33a713cc42c9be05e4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:38.316175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:38.316735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:38.317099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:38.338864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:38.338864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:38.410005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1106/2214] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:38.420391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:38.422477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:38.537361 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7fad6-b2e5-4fde-9355-baacc2f64be4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:38.544302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:38.554410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:38.555243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:38.555619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:38.590264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:38.596478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5eea2-67ad-4539-9c40-d11518b9b503 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:38.620863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1109/2215] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:39.495725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:39.497429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:39.513280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da1835-eb81-4c04-8aa1-635e6f69337b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:39.519744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:39.521685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:39.522097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:39.522578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] PATH_INFO: `/v3/projects/30efec78809b4a9f962f6b866e41d5f5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:39.524279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30efec78809b4a9f962f6b866e41d5f5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:39.530206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:39.583121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:39.610040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30efec78809b4a9f962f6b866e41d5f5', 'request_id': 'req-dd3f5888-b5ff-4218-8665-9c0d25945905'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:39.623306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30efec78809b4a9f962f6b866e41d5f5', 'request_id': 'req-dd3f5888-b5ff-4218-8665-9c0d25945905'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:39.634446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30efec78809b4a9f962f6b866e41d5f5', 'request_id': 'req-dd3f5888-b5ff-4218-8665-9c0d25945905'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:39.657261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] The token cache is being invalidate because project 30efec78809b4a9f962f6b866e41d5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:10:39.657464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:39.658263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd3f5888-b5ff-4218-8665-9c0d25945905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:10:39.660654 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1107/2216] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:10:39 2022] DELETE /identity/v3/projects/30efec78809b4a9f962f6b866e41d5f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:39.665222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:39.665730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:39.665823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:39.699476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:39.701319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:39.839000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1110/2217] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:39.845645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:39.845935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:39.846728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:39.876019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:39.877265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:39.896230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1108/2218] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:39.900716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:39.916218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151baef1-a826-4e10-9305-089fcf4a4a3c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:39.933766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:39.933766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:39.933766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:39.933766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:39.934491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:10:39.934491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:39.935313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:39.966823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1111/2219] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:10:39.974523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:39.998731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df772ebd-b1c0-48d8-9063-59a4b99fbbc9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.052589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.053412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.055708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.056472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.057303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:40.061173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:40.123206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1109/2220] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:10:39 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:40.132272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:40.145143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820cb8-7e47-4172-95ed-14263139f1c3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.280211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:40.345749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.345749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.345749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.345749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.346513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:40.348674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:40.395978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1112/2221] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:10:40 2022] POST /identity/v3/users => generated 287 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:10:40.399824 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:40.417815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b0ed-b558-48bd-aea2-f35bf7548677 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.433771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.437672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.438376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.438950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.439961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:40.445818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:40.477896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:40.498193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1110/2222] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:40.501866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f820f0-0b50-4312-9545-b6e8d25a8e3e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.514107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.514882 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.515170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.515634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.516525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:40.523128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b38744-b43a-41d5-9929-c2ee63819d93 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:40.541636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:40.565335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1113/2223] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:10:40.571346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9512b3b4-25e0-4013-9807-a3f42847d6b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.581139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.582472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.583052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.583511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] PATH_INFO: `/v3/projects/1c5b07c9b6004015bc1e693324d246ce/users/6e8e19cb76464144a74a8504bcc42083/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.608928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b07c9b6004015bc1e693324d246ce, user_id=6e8e19cb76464144a74a8504bcc42083, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:40.612870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:40.642340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1111/2224] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:10:40 2022] PUT /identity/v3/projects/1c5b07c9b6004015bc1e693324d246ce/users/6e8e19cb76464144a74a8504bcc42083/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:10:40.647444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc69d29-3d08-4b6f-bed4-05865f025e7f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.647744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc69d29-3d08-4b6f-bed4-05865f025e7f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.648030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc69d29-3d08-4b6f-bed4-05865f025e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:40.705230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8dc69d29-3d08-4b6f-bed4-05865f025e7f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:40.707600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8dc69d29-3d08-4b6f-bed4-05865f025e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:40.845245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1114/2225] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:40.864788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:40.866992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:40.976611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb8c4b-14f5-47b8-85a7-84a2e2e4688c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:40.984106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:40.985571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:40.986100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:40.986443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:41.025947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:41.033179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:41.047817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1112/2226] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:42.540108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0bbe-b0b2-4d6e-b968-bfd51cce1091 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:42.540683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0bbe-b0b2-4d6e-b968-bfd51cce1091 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:42.541045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb0bbe-b0b2-4d6e-b968-bfd51cce1091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:42.578445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-91eb0bbe-b0b2-4d6e-b968-bfd51cce1091 None None] MFA Rules not processed for user `6e8e19cb76464144a74a8504bcc42083`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:42.581688 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91eb0bbe-b0b2-4d6e-b968-bfd51cce1091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:42.667601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1115/2227] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:42.680947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:42.682408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:42.744453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0668fc-061d-407d-a65f-f7f3f9efc5c0 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:42.748756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:42.749936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:42.750226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:42.750508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:42.759586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:42.762941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:42.764647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:42.775766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1113/2228] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:44.099686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e61e-9ec6-4b05-908a-defdff3c0cb3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:44.100131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e61e-9ec6-4b05-908a-defdff3c0cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:44.100256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e61e-9ec6-4b05-908a-defdff3c0cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:44.130116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8965e61e-9ec6-4b05-908a-defdff3c0cb3 None None] MFA Rules not processed for user `af8f9521b12d464cbd49422a77842b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:44.131513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8965e61e-9ec6-4b05-908a-defdff3c0cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:44.171388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:44.232895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1116/2229] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:44.252357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:10:44.253950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:44.340094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18073-0d71-48f0-aca7-ff74a301235e service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:10:44.345745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:10:44.346402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:44.346688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:44.347068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:44.348928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:44.422789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:10:44.424559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e888c1-52a4-49af-81cd-1666f89c63a1 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:10:44.435047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1114/2230] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:10:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:10:45.393083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69311ca-5c1a-4dfe-879d-444865eec1ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:45.394173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69311ca-5c1a-4dfe-879d-444865eec1ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:45.394634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69311ca-5c1a-4dfe-879d-444865eec1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:45.415726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d69311ca-5c1a-4dfe-879d-444865eec1ba None None] MFA Rules not processed for user `6e8e19cb76464144a74a8504bcc42083`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:45.417349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d69311ca-5c1a-4dfe-879d-444865eec1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:45.430841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1117/2231] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:10:45.837262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:45.838237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:45.838533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:45.865030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] MFA Rules not processed for user `0a36e02c916b494180e68d3f231ee714`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:45.866834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:45.876519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1115/2232] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:55.409548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=17 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:10:55.583645 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb70f4-a63c-4473-b91f-822f9e0bad88 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:10:55.583935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb70f4-a63c-4473-b91f-822f9e0bad88 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:10:55.584213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb70f4-a63c-4473-b91f-822f9e0bad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:10:55.620192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99eb70f4-a63c-4473-b91f-822f9e0bad88 None None] MFA Rules not processed for user `4549c33736bc413a87460db0c43f5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:10:55.621387 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99eb70f4-a63c-4473-b91f-822f9e0bad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:10:55.697742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1118/2233] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:10:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:10:55.860085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=61 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:05.695619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=37 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:19.096679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:19.098607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:19.197940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ce55d-5d72-41ae-92e9-9f90db9cb682 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:19.203602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:19.204523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:19.204796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:19.205066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] PATH_INFO: `/v3/users/81c86423d6a645fe8fe8062daed5fcdd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:19.206333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c86423d6a645fe8fe8062daed5fcdd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:19.208503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:19.271070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c86423d6a645fe8fe8062daed5fcdd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.279277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c86423d6a645fe8fe8062daed5fcdd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.279926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c86423d6a645fe8fe8062daed5fcdd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.293153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c86423d6a645fe8fe8062daed5fcdd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.293753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c86423d6a645fe8fe8062daed5fcdd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.306547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1116/2234] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:19 2022] DELETE /identity/v3/users/81c86423d6a645fe8fe8062daed5fcdd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:19.311621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:19.312231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:19.312231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:19.340598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:19.341805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:19.402067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1119/2235] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:19.411690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:19.413073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:19.488448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:19.512897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3b61d-7564-49ab-bf57-06ec8f2a4d5f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:19.516340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:19.517667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:19.517945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:19.518216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:19.530062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:19.531541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8da490-d2da-4ca0-b022-7600d5ff404c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:19.541319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1117/2236] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:19.821545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:19.824252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:19.836944 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a1867-66ed-41f6-a092-78a3dc8c0c7c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:19.841923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:19.843082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:19.843437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:19.844179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] PATH_INFO: `/v3/projects/2c9d3e291460490496762f44871efac4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:19.845341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9d3e291460490496762f44871efac4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:19.847274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:19.900243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9d3e291460490496762f44871efac4', 'request_id': 'req-5d79442e-f078-4f0e-b5b2-07044563c833'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.906331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9d3e291460490496762f44871efac4', 'request_id': 'req-5d79442e-f078-4f0e-b5b2-07044563c833'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.913892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9d3e291460490496762f44871efac4', 'request_id': 'req-5d79442e-f078-4f0e-b5b2-07044563c833'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.926011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] The token cache is being invalidate because project 2c9d3e291460490496762f44871efac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:19.926301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.926975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d79442e-f078-4f0e-b5b2-07044563c833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:19.928574 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1120/2237] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:19 2022] DELETE /identity/v3/projects/2c9d3e291460490496762f44871efac4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:19.933619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:19.933991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:19.934611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.020137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:20.021392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:20.084329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1118/2238] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:20.155576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.155897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.156209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.176638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:20.178008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:20.194248 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1121/2239] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:20.198132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.209402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c5c93-6086-4467-8b3d-4946632d840e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.220899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.222095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.222916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.223500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.224438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:11:20.224928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.226909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.248196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1119/2240] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:11:20.254104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.265403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e64911-31e7-4528-babb-b46fa7960ef6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.271600 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.272258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.272541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.272865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.273551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.275572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.319824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1122/2241] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:20 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:20.323778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.334627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8315e4e-4cd9-4de9-909f-b99f53a3e139 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.400166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.401312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.401312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.401520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.402295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.404695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.429088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1120/2242] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:20 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:20.432846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.451397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92283b1a-f382-46fe-a60e-d23b2c8d6f4c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.458082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.459392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.460244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.460579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.461505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.463820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.482806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1123/2243] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:20.486595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.498469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8567f-0664-4a68-bf2a-d5f6c0cb0392 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.503918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.504593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.504873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.505149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.505935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.507585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cd763-d357-4f64-8b31-38e4fdebe497 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.517621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1121/2244] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:20.521813 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.531796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa4dc7-50b0-429f-8c1d-661dac740aa5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.536733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.537676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.537989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.538337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] PATH_INFO: `/v3/projects/6dd3241e9f134a7bb89bbaf40232959c/users/1957cb1501da480f9974bda730ce9db3/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.553367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd3241e9f134a7bb89bbaf40232959c, user_id=1957cb1501da480f9974bda730ce9db3, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.556054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.569856 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1124/2245] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:20 2022] PUT /identity/v3/projects/6dd3241e9f134a7bb89bbaf40232959c/users/1957cb1501da480f9974bda730ce9db3/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:20.575136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07fbf-4d0d-4844-b7fa-54dcc1eef404 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.575449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07fbf-4d0d-4844-b7fa-54dcc1eef404 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.575742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf07fbf-4d0d-4844-b7fa-54dcc1eef404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.594971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9cf07fbf-4d0d-4844-b7fa-54dcc1eef404 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:20.596642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9cf07fbf-4d0d-4844-b7fa-54dcc1eef404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:20.609171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:20.656867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1122/2246] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:20.666982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:20.668663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:20.682945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:20.725978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21f8c-aa22-48dd-a5f0-b19bfa45abef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:20.729629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:20.730267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:20.730552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:20.730860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:20.739842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:20.741268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:20.750631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1125/2247] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:21.805904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c704f-da07-4efe-a4c4-5b6dd13a5cb2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:21.806342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c704f-da07-4efe-a4c4-5b6dd13a5cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:21.806342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c704f-da07-4efe-a4c4-5b6dd13a5cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:21.843798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c85c704f-da07-4efe-a4c4-5b6dd13a5cb2 None None] MFA Rules not processed for user `1957cb1501da480f9974bda730ce9db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:21.845134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c85c704f-da07-4efe-a4c4-5b6dd13a5cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:21.918306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1123/2248] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:21.930054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:21.931566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:22.011893 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b757311-c161-4c60-9524-2ab32c79b996 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:22.015676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:22.016320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:22.016804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:22.017229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:22.025920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:22.027392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff964da0-05dd-463f-926b-1557869fc51c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:22.036725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1126/2249] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:24.465561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:24.466052 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:24.466148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:24.483261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] MFA Rules not processed for user `1957cb1501da480f9974bda730ce9db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:24.484605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:24.491408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1124/2250] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:25.989585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:25.990098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:25.990193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:26.007708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] MFA Rules not processed for user `6e8e19cb76464144a74a8504bcc42083`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:26.008909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:26.066648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1127/2251] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:29.336435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:29.337863 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:29.340567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:29.341979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:29.397629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:29.454941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e479a-58fb-4533-9a41-5ca42612fe16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:29.455202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18747825-0e39-4b52-a719-f6b6cb172031 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:29.461707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:29.462531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:29.462846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:29.463128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:29.463434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:29.464642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:29.465109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:29.465243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:29.465703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:29.469329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:29.482153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:29.484105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:29.487163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:29.488624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff22c26-9e50-4c4f-ac05-6bafde6acffc service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:29.495489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1128/2252] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:29.499337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1125/2253] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:33.572308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:33.574358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:33.588301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1968337-799a-4ee7-b860-f5b606bf4cb1 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:33.593226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26347288-cec7-4074-8754-4175164135db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:33.594108 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347288-cec7-4074-8754-4175164135db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:33.594373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347288-cec7-4074-8754-4175164135db admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:33.594642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26347288-cec7-4074-8754-4175164135db admin admin] PATH_INFO: `/v3/users/40812967594143268681b6952c450768` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:33.595866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26347288-cec7-4074-8754-4175164135db admin admin] RBAC: Authorizing `identity:delete_user(user_id=40812967594143268681b6952c450768)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:33.599353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26347288-cec7-4074-8754-4175164135db admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:33.659396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26347288-cec7-4074-8754-4175164135db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40812967594143268681b6952c450768'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:33.674795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26347288-cec7-4074-8754-4175164135db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40812967594143268681b6952c450768'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:33.675604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26347288-cec7-4074-8754-4175164135db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40812967594143268681b6952c450768'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:33.695354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26347288-cec7-4074-8754-4175164135db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40812967594143268681b6952c450768'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:33.696279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-26347288-cec7-4074-8754-4175164135db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40812967594143268681b6952c450768'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:33.716925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1129/2254] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:33 2022] DELETE /identity/v3/users/40812967594143268681b6952c450768 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:33.719434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:33.719933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:33.720314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:33.740606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:33.742049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:33.751331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:33.819940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1126/2255] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:33.832997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26347288-cec7-4074-8754-4175164135db admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:33.834591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26347288-cec7-4074-8754-4175164135db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:33.900103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26347288-cec7-4074-8754-4175164135db admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:33.904249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:33.905041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:33.905201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:33.905480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:33.919502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:33.921087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:33.931820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1130/2256] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:34.216450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:34.218173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:34.231596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c33d89-7447-488d-a493-fd7d9b245477 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:34.248966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:34.250653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:34.251013 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:34.251424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] PATH_INFO: `/v3/projects/563b299739474e9bb1e9b52ffb54f92d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:34.253735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=563b299739474e9bb1e9b52ffb54f92d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:34.256114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:34.316594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '563b299739474e9bb1e9b52ffb54f92d', 'request_id': 'req-320cc417-02fc-4b1b-91e4-0cf3beddf2af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.325692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '563b299739474e9bb1e9b52ffb54f92d', 'request_id': 'req-320cc417-02fc-4b1b-91e4-0cf3beddf2af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.336656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '563b299739474e9bb1e9b52ffb54f92d', 'request_id': 'req-320cc417-02fc-4b1b-91e4-0cf3beddf2af'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.349855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] The token cache is being invalidate because project 563b299739474e9bb1e9b52ffb54f92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:34.350440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.351123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.353553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1127/2257] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:34 2022] DELETE /identity/v3/projects/563b299739474e9bb1e9b52ffb54f92d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:34.357816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:34.368117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477faa-1a5d-41e8-b973-13a9fd9d7012 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:34.397485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:34.431929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:34.432913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:34.433259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:34.433603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] PATH_INFO: `/v3/users/f83bb0ac9f59427c94c4b03374ddd7e9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:34.435435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83bb0ac9f59427c94c4b03374ddd7e9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:34.437236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:34.492975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83bb0ac9f59427c94c4b03374ddd7e9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.501768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83bb0ac9f59427c94c4b03374ddd7e9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.503147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83bb0ac9f59427c94c4b03374ddd7e9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.515900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83bb0ac9f59427c94c4b03374ddd7e9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.516560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a032bca-ac0d-419e-903c-63d0cd10b209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83bb0ac9f59427c94c4b03374ddd7e9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.529326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1131/2258] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:34 2022] DELETE /identity/v3/users/f83bb0ac9f59427c94c4b03374ddd7e9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:34.823349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:34.824912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:34.849705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cc417-02fc-4b1b-91e4-0cf3beddf2af admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:34.919250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:34.920184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:34.920534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:34.920873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] PATH_INFO: `/v3/projects/efdc9b3337e6413cadc9b696bbb848f1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:34.922144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efdc9b3337e6413cadc9b696bbb848f1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:34.924164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:34.965507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:34.975044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efdc9b3337e6413cadc9b696bbb848f1', 'request_id': 'req-ef4e6a20-8781-44bb-b909-9be4021476a1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.982935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efdc9b3337e6413cadc9b696bbb848f1', 'request_id': 'req-ef4e6a20-8781-44bb-b909-9be4021476a1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:34.990404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efdc9b3337e6413cadc9b696bbb848f1', 'request_id': 'req-ef4e6a20-8781-44bb-b909-9be4021476a1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:35.002083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] The token cache is being invalidate because project efdc9b3337e6413cadc9b696bbb848f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:35.002369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:35.003082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ef4e6a20-8781-44bb-b909-9be4021476a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:35.004804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1128/2259] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:34 2022] DELETE /identity/v3/projects/efdc9b3337e6413cadc9b696bbb848f1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:35.009685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.010163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.010261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.030294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:35.031557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:35.100543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1132/2260] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:11:35.108287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.108928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.109670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.133942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:35.135200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:35.143440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1129/2261] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:35.147895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.167023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf69eb5-90ee-4d9d-bd6f-ba908751855e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.171911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.172584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.172866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.173165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.173897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:11:35.174904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.176531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.191239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1133/2262] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:11:35.196842 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.207834 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c00aa-00d3-425f-a448-c5eb3534bed1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.212737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.222612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.222612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.222612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.222612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.222612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.256150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1130/2263] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:35.257542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.269896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679e72-88f7-4d50-918d-be3a94d8562a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.308164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:35.339570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.340238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.340565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.340837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.341518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.343525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.368930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1134/2264] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:35.370582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.383038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165b3c0-56b5-4503-8fb6-618e5c3be723 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.388673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.389340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.389619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.389892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.390622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.392427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.407420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1131/2265] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:35.409249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.424358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69914125-a383-4325-be82-2d53d9d11ad4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.429328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.430007 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.430291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.430568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.431339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.432971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba560a86-02d4-4669-885d-a7980435da89 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.445401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1135/2266] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:35.450572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.469287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331fab0-7a7f-4a48-ac47-ec501f10d400 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.477601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.479421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.479421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.479421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] PATH_INFO: `/v3/projects/35e9dfa91bab41fc853491b03ed15da2/users/1814017afbd348bb806de423752460d1/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.504392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e9dfa91bab41fc853491b03ed15da2, user_id=1814017afbd348bb806de423752460d1, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.508619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.525374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1132/2267] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:35 2022] PUT /identity/v3/projects/35e9dfa91bab41fc853491b03ed15da2/users/1814017afbd348bb806de423752460d1/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:35.537546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af889-0c26-49bb-8429-17559fc69852 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.537899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af889-0c26-49bb-8429-17559fc69852 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.538308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368af889-0c26-49bb-8429-17559fc69852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.580037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-368af889-0c26-49bb-8429-17559fc69852 None None] MFA Rules not processed for user `1814017afbd348bb806de423752460d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:35.581815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-368af889-0c26-49bb-8429-17559fc69852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:35.663303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1136/2268] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:35.667762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.738356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2a586-feb5-4d84-9d4e-8235d23e75ec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.743000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.743658 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.743939 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.744214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.744905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.746849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.773820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1133/2269] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:35.777737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af889-0c26-49bb-8429-17559fc69852 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.788799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368af889-0c26-49bb-8429-17559fc69852 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.847877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.848559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.848851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.849128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.849833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.851932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.874746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1137/2270] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:35 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:35.876003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.887143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e881a-d20a-4772-9165-33d66e5a5159 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.892667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.893876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.894193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.894472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.895308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.896920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.908469 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1134/2271] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:35.912578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.923256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d198d-8732-4b07-9a3c-61ca4fad9528 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:35.929333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:35.931487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:35.931487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:35.931711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] PATH_INFO: `/v3/projects/c36f79d25951459386fdfb11eac13926/users/33fa5c0d17764fd691e567246a1e0ee5/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:35.953239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36f79d25951459386fdfb11eac13926, user_id=33fa5c0d17764fd691e567246a1e0ee5, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:35.956588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:35.969820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1138/2272] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:35 2022] PUT /identity/v3/projects/c36f79d25951459386fdfb11eac13926/users/33fa5c0d17764fd691e567246a1e0ee5/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:35.973686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:35.983792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84d70-1cb5-42b0-9a60-85db0861b7b2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:36.019643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:36.046648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:36.047352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.047645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.047923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.048653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:36.050813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb9b1f-a985-4bc2-abcf-9ef5cc716e38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:36.060878 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1135/2273] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:36.065504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:36.075790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eb0ae-a295-4bd9-b774-04a47f561f97 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:36.080402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:36.081284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.081584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.081841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] PATH_INFO: `/v3/projects/c36f79d25951459386fdfb11eac13926/users/33fa5c0d17764fd691e567246a1e0ee5/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.083854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36f79d25951459386fdfb11eac13926, user_id=33fa5c0d17764fd691e567246a1e0ee5, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:36.086474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:36.089565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:36.099230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1139/2274] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:36 2022] PUT /identity/v3/projects/c36f79d25951459386fdfb11eac13926/users/33fa5c0d17764fd691e567246a1e0ee5/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:11:36.109361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.109662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.109944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.137594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] MFA Rules not processed for user `33fa5c0d17764fd691e567246a1e0ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:36.138624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:36.212153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1136/2275] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:36.225667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:36.227692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:36.289631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fbba0-d814-42ac-9fdb-d87dbbdf8e88 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:36.293472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:36.294131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.294414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.294687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.308913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:36.310945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b533cb4-9634-436f-944d-f589821a6b36 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:36.322571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1140/2276] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:36.397296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:36.464518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce58026-43ed-4202-be80-858ac4079ecb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:36.469118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:36.469991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.470276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.470581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] PATH_INFO: `/v3/users/1814017afbd348bb806de423752460d1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.472454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1814017afbd348bb806de423752460d1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:36.474420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:36.535231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1814017afbd348bb806de423752460d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:36.542689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1814017afbd348bb806de423752460d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:36.543335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1814017afbd348bb806de423752460d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:36.551477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:36.557439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1814017afbd348bb806de423752460d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:36.558245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1814017afbd348bb806de423752460d1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:36.585384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1137/2277] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:36 2022] DELETE /identity/v3/users/1814017afbd348bb806de423752460d1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:36.587104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.587396 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.587675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.610633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:36.612324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:36.682782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1141/2278] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:36.692508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:36.694010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:36.766447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53ea3-3959-4114-98b9-12d01eb58d23 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:36.770891 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:36.771564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:36.771846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:36.772420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:36.784687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:36.786446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:36.798211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1138/2279] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:37.176493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:37.178057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:37.195539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cda92-f8bf-416d-8e44-7d3bf794cba1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:37.200442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:37.201311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:37.201593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:37.201873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] PATH_INFO: `/v3/projects/35e9dfa91bab41fc853491b03ed15da2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:37.203097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e9dfa91bab41fc853491b03ed15da2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:37.204999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:37.252951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e9dfa91bab41fc853491b03ed15da2', 'request_id': 'req-1fe40090-112f-4620-91be-3ea0d5f83934'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.258870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e9dfa91bab41fc853491b03ed15da2', 'request_id': 'req-1fe40090-112f-4620-91be-3ea0d5f83934'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.265430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e9dfa91bab41fc853491b03ed15da2', 'request_id': 'req-1fe40090-112f-4620-91be-3ea0d5f83934'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.276553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] The token cache is being invalidate because project 35e9dfa91bab41fc853491b03ed15da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:37.276841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.277745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.279307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1142/2280] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:37 2022] DELETE /identity/v3/projects/35e9dfa91bab41fc853491b03ed15da2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:37.283592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:37.293728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ed8ce-f7d3-4626-aa70-5014e6573a95 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:37.352088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:37.352945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:37.353222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:37.353491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] PATH_INFO: `/v3/users/33fa5c0d17764fd691e567246a1e0ee5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:37.354670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fa5c0d17764fd691e567246a1e0ee5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:37.356601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:37.401194 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fa5c0d17764fd691e567246a1e0ee5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.408047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fa5c0d17764fd691e567246a1e0ee5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.408669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fa5c0d17764fd691e567246a1e0ee5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.420151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fa5c0d17764fd691e567246a1e0ee5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.420786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-94bc37c1-9636-4d67-9562-939bde6e7704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fa5c0d17764fd691e567246a1e0ee5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.428898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:37.433827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1139/2281] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:37 2022] DELETE /identity/v3/users/33fa5c0d17764fd691e567246a1e0ee5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:37.725693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:37.728556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:37.757586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe40090-112f-4620-91be-3ea0d5f83934 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:37.787161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:37.839634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:37.840505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:37.840794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:37.841075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] PATH_INFO: `/v3/projects/c36f79d25951459386fdfb11eac13926` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:37.842321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36f79d25951459386fdfb11eac13926)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:37.844283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:37.891001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36f79d25951459386fdfb11eac13926', 'request_id': 'req-21c4a854-e512-48c2-b057-e2c45b184a0d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.897691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36f79d25951459386fdfb11eac13926', 'request_id': 'req-21c4a854-e512-48c2-b057-e2c45b184a0d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.904212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36f79d25951459386fdfb11eac13926', 'request_id': 'req-21c4a854-e512-48c2-b057-e2c45b184a0d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.915038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] The token cache is being invalidate because project c36f79d25951459386fdfb11eac13926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:37.915328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.915954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21c4a854-e512-48c2-b057-e2c45b184a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:37.917494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1143/2282] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:37 2022] DELETE /identity/v3/projects/c36f79d25951459386fdfb11eac13926 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:37.922442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:37.922950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:37.923425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:37.945110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:37.947060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:38.008566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1140/2283] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:38.017556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.018462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.018745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.037071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:38.038338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:38.046285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1144/2284] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:38.050445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.060224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdc4e5-ae29-4113-aa46-1f69f03860b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.065266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.065956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.066246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.066526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.067300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:11:38.067610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.069206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.081378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1141/2285] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:11:38.085647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.094937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b2df4-8ce1-4da4-8ba7-70f9395e3f5d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.099646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.100296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.100583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.100867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.101548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.103493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.128870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1145/2286] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:38.132931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.143558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969a819-7765-427d-acea-cd99c49e6ad0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.201007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.201712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.202003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.202314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.203058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.204975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.228564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1142/2287] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:38.232621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.250603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f80d2-9a67-42e6-9cc2-7293d29158ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.256041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.256708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.257051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.257335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.258066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.259769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.271365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1146/2288] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:38.275558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.285794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353b094-969b-41d1-a9e0-168006d22ec6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.290677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.291310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.291596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.291875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.292602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.294202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7f58b-ff58-4222-b140-2496583a372e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.304708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1143/2289] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:38.308845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.319895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce91dc-c93f-4477-9eb3-aecc091372cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.324568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.325487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.325750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.326026 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] PATH_INFO: `/v3/projects/f061709136e04865af09048cedb42965/users/65575092aa79445da5b5016bda67a62a/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.341328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f061709136e04865af09048cedb42965, user_id=65575092aa79445da5b5016bda67a62a, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.344031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.356559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1147/2290] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:38 2022] PUT /identity/v3/projects/f061709136e04865af09048cedb42965/users/65575092aa79445da5b5016bda67a62a/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:38.367320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.367644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.367929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.394354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] MFA Rules not processed for user `65575092aa79445da5b5016bda67a62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:38.395602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:38.449353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:38.459016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1144/2291] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:38.463448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.512398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:38.527049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d2ce7-f10e-413b-b70f-4e0be960b451 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.531640 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.532286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.532567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.532839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.533547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.535465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.560041 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1148/2292] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:38.563924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.575254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94143ce-e134-4760-8d8c-5f54568b2901 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.630571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.631271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.631559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.631839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.632532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.634455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.654093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1145/2293] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:38.657986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.668656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd8bd-ed31-4dbc-aa73-ff60fd13b55f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.673425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.674083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.674392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.674690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.675456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.677050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.687779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1149/2294] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:11:38.692211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.701810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b403cc-0a51-401f-a38c-52284b494a5e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.706993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.707859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.708143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.708418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] PATH_INFO: `/v3/projects/c90cc993e09a4fe3be75d65f22080d35/users/00cee1149b184b40b46ac32169d40517/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.722924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90cc993e09a4fe3be75d65f22080d35, user_id=00cee1149b184b40b46ac32169d40517, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.725571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.737817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1146/2295] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:38 2022] PUT /identity/v3/projects/c90cc993e09a4fe3be75d65f22080d35/users/00cee1149b184b40b46ac32169d40517/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:11:38.741642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.751567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ea855-143d-4d18-8e5f-8bfe44fa93d3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.809091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.810151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.810388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.810668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.811432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.813047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc178b-c636-4c8b-8414-fcf935d63958 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.827672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1150/2296] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:38.828096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:38.839710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04001ca0-eaf6-46b8-bbb0-815d71153329 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:38.848302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:38.849199 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.849481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.849757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] PATH_INFO: `/v3/projects/c90cc993e09a4fe3be75d65f22080d35/users/00cee1149b184b40b46ac32169d40517/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.852696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90cc993e09a4fe3be75d65f22080d35, user_id=00cee1149b184b40b46ac32169d40517, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:38.856222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:38.885695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1147/2297] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:38 2022] PUT /identity/v3/projects/c90cc993e09a4fe3be75d65f22080d35/users/00cee1149b184b40b46ac32169d40517/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:11:38.896968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f791a9-e827-4631-9d72-81714352c532 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:38.897256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f791a9-e827-4631-9d72-81714352c532 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:38.897540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f791a9-e827-4631-9d72-81714352c532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:38.927311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-20f791a9-e827-4631-9d72-81714352c532 None None] MFA Rules not processed for user `00cee1149b184b40b46ac32169d40517`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:38.928531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-20f791a9-e827-4631-9d72-81714352c532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:39.007958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1151/2298] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:39.020369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:39.022114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:39.078238 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fc18-6fed-40a4-a746-4b2d32e0007e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:39.082374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:39.083039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:39.083321 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:39.083600 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:39.092614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:39.094035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f540d-eb84-4957-8ac9-ea9044ac2a75 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:39.103884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1148/2299] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:39.227967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f791a9-e827-4631-9d72-81714352c532 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:39.272158 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:39.290606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f791a9-e827-4631-9d72-81714352c532 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:39.295105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:39.295970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:39.296252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:39.296522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] PATH_INFO: `/v3/users/65575092aa79445da5b5016bda67a62a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:39.297728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65575092aa79445da5b5016bda67a62a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:39.299732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:39.345196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65575092aa79445da5b5016bda67a62a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.352353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65575092aa79445da5b5016bda67a62a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.352999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65575092aa79445da5b5016bda67a62a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.366257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65575092aa79445da5b5016bda67a62a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.366945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65575092aa79445da5b5016bda67a62a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.379643 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1152/2300] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:39 2022] DELETE /identity/v3/users/65575092aa79445da5b5016bda67a62a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:39.384114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:39.384609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:39.384705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:39.402816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:39.404031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:39.448804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:39.465619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1149/2301] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:39.475089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:39.476555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:39.533092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9352f13-aa5a-4f42-8c43-0a7dba0bed17 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:39.536739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:39.537376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:39.537760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:39.538039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:39.546739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:39.548204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:39.557444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1153/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:39.831304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:39.832730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:39.848496 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c5ba7-140a-413b-85ea-269653029d45 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:39.853538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:39.854570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:39.855054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:39.855435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] PATH_INFO: `/v3/projects/f061709136e04865af09048cedb42965` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:39.856938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f061709136e04865af09048cedb42965)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:39.858946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:39.905857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f061709136e04865af09048cedb42965', 'request_id': 'req-e5f5664b-2f22-4b70-aa38-159ca5218ec3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.913812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f061709136e04865af09048cedb42965', 'request_id': 'req-e5f5664b-2f22-4b70-aa38-159ca5218ec3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.920000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f061709136e04865af09048cedb42965', 'request_id': 'req-e5f5664b-2f22-4b70-aa38-159ca5218ec3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.930462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] The token cache is being invalidate because project f061709136e04865af09048cedb42965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:39.930745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.931408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:39.932919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1150/2303] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:39 2022] DELETE /identity/v3/projects/f061709136e04865af09048cedb42965 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:39.937264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:39.947918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d9389-dd4f-4829-8b7c-13fa10aec27b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.007556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.008434 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.008713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.008985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] PATH_INFO: `/v3/users/00cee1149b184b40b46ac32169d40517` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.010179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00cee1149b184b40b46ac32169d40517)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.012118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.017724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:40.068737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00cee1149b184b40b46ac32169d40517'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.079091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00cee1149b184b40b46ac32169d40517'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.079890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00cee1149b184b40b46ac32169d40517'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.094331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00cee1149b184b40b46ac32169d40517'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.095045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-229bb56e-d0f5-46b4-a3f5-82eb1683aae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00cee1149b184b40b46ac32169d40517'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.114227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1154/2304] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:39 2022] DELETE /identity/v3/users/00cee1149b184b40b46ac32169d40517 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:40.384802 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.387754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:40.402817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5664b-2f22-4b70-aa38-159ca5218ec3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.459399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.460252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.460537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.460811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] PATH_INFO: `/v3/projects/c90cc993e09a4fe3be75d65f22080d35` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.462000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90cc993e09a4fe3be75d65f22080d35)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.463932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.509177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90cc993e09a4fe3be75d65f22080d35', 'request_id': 'req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.516529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90cc993e09a4fe3be75d65f22080d35', 'request_id': 'req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.523509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90cc993e09a4fe3be75d65f22080d35', 'request_id': 'req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.533903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] The token cache is being invalidate because project c90cc993e09a4fe3be75d65f22080d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:40.534178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.534849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3b86aa19-c23c-4ff4-85ed-fa3366b9459b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:40.536389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1151/2305] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:40 2022] DELETE /identity/v3/projects/c90cc993e09a4fe3be75d65f22080d35 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:40.541776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.542234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.542332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.562478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:40.563727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:40.631701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1155/2306] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:11:40.640358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.640628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.640914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.649105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:40.661960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:40.663907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:40.672645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1152/2307] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:40.677140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.687206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c484d0-b678-462f-955d-e3eb88e9f1bf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.692417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.693072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.693359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.693642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.694371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:11:40.694681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.696317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.707880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1156/2308] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:11:40.712020 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.721335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c3144-80b4-403e-a437-78208db2307e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.726031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.726671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.726993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.727271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.727967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.729909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.756053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1153/2309] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:40 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:40.759870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.769602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41141fa-3e79-4108-b2eb-97c9ea950bad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.825998 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.826661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.827136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.827526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.828216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.830181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.851061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1157/2310] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:40 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:40.854019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.864790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7b33-de33-440e-9d3c-83a5138cb74a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.869457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.870110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.870394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.870688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.871447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.873036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.883916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1154/2311] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:40.887632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.893290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:40.902119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122d183-0d80-40ca-b4c8-44a828bd5e27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.907647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.908943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.909362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.909671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.910578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.912268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f5066-e326-46b0-9e70-fdf8726494e2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.922547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1158/2312] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:40.927380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:40.937564 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3750f87-5227-4ba2-8f68-efefb4589b8d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:40.942258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:40.943164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.943445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.943720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] PATH_INFO: `/v3/projects/5c36c5c862e34a06aca87895143e2204/users/aaa8615088b54e949710afeaf2642d63/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:40.958030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c36c5c862e34a06aca87895143e2204, user_id=aaa8615088b54e949710afeaf2642d63, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:40.960681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:40.974284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1155/2313] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:40 2022] PUT /identity/v3/projects/5c36c5c862e34a06aca87895143e2204/users/aaa8615088b54e949710afeaf2642d63/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:40.984391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:40.984765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:40.985119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.012500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] MFA Rules not processed for user `aaa8615088b54e949710afeaf2642d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:41.013799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:41.081935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1159/2314] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:41.086342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.148947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b8881-feda-4fd6-bf7c-5557dbb1e078 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.153605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.154258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.154543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.154856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.155569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.157500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.186750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1156/2315] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:41.188049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.199035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205ad8d-13e6-4241-8b37-4c7b8aefbf6e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.259390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.260099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.260392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.260756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.261348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.263319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.295118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1160/2316] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:41.299132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.310067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65351-501e-40f6-aba3-93cec4bc2658 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.314838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.315576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.316001 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.316344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.317152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.318953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.329675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1157/2317] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:11:41.334165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.344212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bc57-02c6-43e8-a638-5af362e7ca51 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.349557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.350227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.350511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.350808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.351555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.353161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015c2f-6129-4391-bff5-dc53281bebde admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.365548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1161/2318] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:41.369695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.380383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7933b4-2eca-4750-8c78-8b568cd7331a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.385370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.386514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.386919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.387265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] PATH_INFO: `/v3/projects/e179e46652614f829343722278e0b981/users/7300c3bb497947b5b411d4d8cd525681/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.403296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e179e46652614f829343722278e0b981, user_id=7300c3bb497947b5b411d4d8cd525681, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.405950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.420559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1158/2319] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:41 2022] PUT /identity/v3/projects/e179e46652614f829343722278e0b981/users/7300c3bb497947b5b411d4d8cd525681/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:41.432254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.432674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.432968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.459907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] MFA Rules not processed for user `7300c3bb497947b5b411d4d8cd525681`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:41.461052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:41.506521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:41.525179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1162/2320] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:41.529731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.581009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:41.593668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b95be9-b105-417d-90f3-e0402d629306 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.599064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.599713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.599995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.600272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.600964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.602885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.629542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1159/2321] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:41.630679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.641493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e1eeb-35e2-4ca3-89c5-86f8a1610fd5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.707978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.708976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.709421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.710384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.711163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.713414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.743572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1163/2322] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:41.747347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.761288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ca0d4-5a46-45cd-b317-92ed979e0082 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.768877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.769736 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.770125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.770604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.772000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.773945 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.790984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1160/2323] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:41.795095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.806360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74da656-8277-4dba-9f81-80e868808d5d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.811708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.812821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.812941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.813128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] PATH_INFO: `/v3/projects/2c413a106433427185c04e097cf6b26f/users/f9c4cf6d958b43f6b32bbe3aa6250a0c/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.833097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c413a106433427185c04e097cf6b26f, user_id=f9c4cf6d958b43f6b32bbe3aa6250a0c, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.835963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.855812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1164/2324] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:41 2022] PUT /identity/v3/projects/2c413a106433427185c04e097cf6b26f/users/f9c4cf6d958b43f6b32bbe3aa6250a0c/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:11:41.859869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.870325 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a33fd-a6ca-4e63-ae08-3b6b6b5e02e1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.927154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.927821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.928101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.928377 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.929129 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.930804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bb296-a6ed-421f-83cb-18ca6616fc52 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.941567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1161/2325] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:11:41.946093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:41.957551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ba21c-d75c-49ef-96cf-262caefa5bb2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:41.962584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:41.963609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.963965 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.964310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] PATH_INFO: `/v3/projects/2c413a106433427185c04e097cf6b26f/users/f9c4cf6d958b43f6b32bbe3aa6250a0c/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:41.966406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c413a106433427185c04e097cf6b26f, user_id=f9c4cf6d958b43f6b32bbe3aa6250a0c, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:41.969199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:41.981377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1165/2326] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:41 2022] PUT /identity/v3/projects/2c413a106433427185c04e097cf6b26f/users/f9c4cf6d958b43f6b32bbe3aa6250a0c/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:41.991985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:41.992281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:41.992562 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:42.022601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] MFA Rules not processed for user `f9c4cf6d958b43f6b32bbe3aa6250a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:42.023890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:42.099231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1162/2327] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:42.110817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:42.113135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:42.173130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1587fb8-0abd-441d-8c80-7367ed37f330 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:42.177209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:42.178731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:42.179204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:42.179572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:42.190066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:42.191571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:42.201553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1166/2328] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:42.244759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:42.246158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:42.258239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c08d2c-0f7d-4ba9-834c-f891fb22904f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:42.262023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:42.262664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:42.262983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:42.263260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:42.314685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:42.327779 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:42.330038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06824a33-1d04-4fbb-9915-c0abeba3b029 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:42.340102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1163/2329] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:45.246146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:45.295680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:45.315464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d4d8c-b90a-4ac7-ba46-51ceefe28595 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:45.320134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:45.321003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:45.321372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:45.321651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] PATH_INFO: `/v3/users/aaa8615088b54e949710afeaf2642d63` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:45.322875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa8615088b54e949710afeaf2642d63)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:45.324795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:45.373647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa8615088b54e949710afeaf2642d63'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:45.381855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa8615088b54e949710afeaf2642d63'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:45.382482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa8615088b54e949710afeaf2642d63'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:45.395012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa8615088b54e949710afeaf2642d63'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:45.395667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa8615088b54e949710afeaf2642d63'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:45.408119 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1167/2330] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:45 2022] DELETE /identity/v3/users/aaa8615088b54e949710afeaf2642d63 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:45.413214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:45.413692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:45.413796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:45.431493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:45.432701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:45.495786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1164/2331] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:11:45.506671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:45.508137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:45.570895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456a06-5b45-493a-abe9-877df609cd91 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:45.574570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:45.575280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:45.575565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:45.575840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:45.587668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:45.589651 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:45.612682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1168/2332] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:45.940671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:45.942289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:45.965305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9aa72-816c-4eca-aa37-418120530f16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:45.972545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:45.973463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:45.973738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:45.973977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] PATH_INFO: `/v3/projects/5c36c5c862e34a06aca87895143e2204` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:45.975718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c36c5c862e34a06aca87895143e2204)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:45.977650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:46.026044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c36c5c862e34a06aca87895143e2204', 'request_id': 'req-865f91e3-f717-4de4-ae59-36d119a98502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.033039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c36c5c862e34a06aca87895143e2204', 'request_id': 'req-865f91e3-f717-4de4-ae59-36d119a98502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.039163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c36c5c862e34a06aca87895143e2204', 'request_id': 'req-865f91e3-f717-4de4-ae59-36d119a98502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.049571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] The token cache is being invalidate because project 5c36c5c862e34a06aca87895143e2204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:46.049852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.050471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.052004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1165/2333] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:45 2022] DELETE /identity/v3/projects/5c36c5c862e34a06aca87895143e2204 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:46.055881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:46.066493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b629-aae7-4f05-b0b7-3d3e0aeedcee service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:46.128125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:46.129100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:46.129447 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:46.129789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] PATH_INFO: `/v3/users/7300c3bb497947b5b411d4d8cd525681` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:46.131091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7300c3bb497947b5b411d4d8cd525681)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:46.133233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:46.140195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:46.191707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7300c3bb497947b5b411d4d8cd525681'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.214622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7300c3bb497947b5b411d4d8cd525681'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.215470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7300c3bb497947b5b411d4d8cd525681'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.229760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7300c3bb497947b5b411d4d8cd525681'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.230441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7300c3bb497947b5b411d4d8cd525681'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.243816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1169/2334] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:46 2022] DELETE /identity/v3/users/7300c3bb497947b5b411d4d8cd525681 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:46.575221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:46.577303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:46.591004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f91e3-f717-4de4-ae59-36d119a98502 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:46.663663 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:46.664512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:46.664792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:46.665186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] PATH_INFO: `/v3/projects/e179e46652614f829343722278e0b981` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:46.666375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e179e46652614f829343722278e0b981)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:46.668304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:46.678877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:46.721844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e179e46652614f829343722278e0b981', 'request_id': 'req-0d242ec6-1395-4307-bae2-6bf04bf73414'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.729168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e179e46652614f829343722278e0b981', 'request_id': 'req-0d242ec6-1395-4307-bae2-6bf04bf73414'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.736468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e179e46652614f829343722278e0b981', 'request_id': 'req-0d242ec6-1395-4307-bae2-6bf04bf73414'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.746467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] The token cache is being invalidate because project e179e46652614f829343722278e0b981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:46.746995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.747656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.749178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1166/2335] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:46 2022] DELETE /identity/v3/projects/e179e46652614f829343722278e0b981 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:46.753204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:46.754622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:46.768918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec778a-9a35-46d9-9c07-7e1edf47f758 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:46.826478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:46.827383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:46.827664 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:46.827935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] PATH_INFO: `/v3/users/f9c4cf6d958b43f6b32bbe3aa6250a0c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:46.829316 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c4cf6d958b43f6b32bbe3aa6250a0c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:46.831935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:46.891227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c4cf6d958b43f6b32bbe3aa6250a0c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.902710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c4cf6d958b43f6b32bbe3aa6250a0c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.903398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c4cf6d958b43f6b32bbe3aa6250a0c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.916946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c4cf6d958b43f6b32bbe3aa6250a0c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.917612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4c2a8cf0-016c-40fc-ad55-965f69e9d6e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c4cf6d958b43f6b32bbe3aa6250a0c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:46.930741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1170/2336] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:11:46 2022] DELETE /identity/v3/users/f9c4cf6d958b43f6b32bbe3aa6250a0c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:47.222204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.223817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:47.246118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d242ec6-1395-4307-bae2-6bf04bf73414 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.312111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.313672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.314146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.315250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] PATH_INFO: `/v3/projects/2c413a106433427185c04e097cf6b26f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.316519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c413a106433427185c04e097cf6b26f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.318917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.365161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c413a106433427185c04e097cf6b26f', 'request_id': 'req-264521b2-bb74-42d9-918c-1ce950216938'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:47.372422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c413a106433427185c04e097cf6b26f', 'request_id': 'req-264521b2-bb74-42d9-918c-1ce950216938'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:47.379231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c413a106433427185c04e097cf6b26f', 'request_id': 'req-264521b2-bb74-42d9-918c-1ce950216938'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:47.389690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] The token cache is being invalidate because project 2c413a106433427185c04e097cf6b26f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:11:47.389976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:47.390619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-264521b2-bb74-42d9-918c-1ce950216938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:11:47.392188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1167/2337] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:11:47 2022] DELETE /identity/v3/projects/2c413a106433427185c04e097cf6b26f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:47.398062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.398348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.398621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.405687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:47.417598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:47.418941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:47.482395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1171/2338] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:47.491419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.491722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.491999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.510595 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:47.511749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:47.519538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1168/2339] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:47.523644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.533314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35b911-dbae-45b3-9de0-3cad52a7ec8b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.538054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.538711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.539022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.539303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.540037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:11:47.540349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.541979 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.554342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1172/2340] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:11:47.558217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.568126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9aec5-ceac-40a8-bf1b-e6c1134a5e42 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.572648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.573286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.573566 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.573840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.574524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.576438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.597114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:47.608644 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1169/2341] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:11:47 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:11:47.609837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.620588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a761c3-3a0d-4228-929d-f345433b717b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.679346 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.680017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.680298 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.680576 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.681260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.683215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.701409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1173/2342] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:11:47 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:11:47.705641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.715961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78bc6-468c-4100-ab0c-4fdceeeb6e83 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.723310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.723976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.724400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.724667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.725547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.727437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.746899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1170/2343] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:11:47.748446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.759075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608e4d1-0764-4b39-9480-2d1fe9bb6b67 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.764210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.764811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.765096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.765374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.766121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.767760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9eb700-f064-4ff3-99de-ae8193a05945 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.779025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1174/2344] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:11:47.783731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:47.793981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fadc1-3a23-4a08-95f1-8f8a6c0e21dd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:47.798680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:47.800178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.800461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.800738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] PATH_INFO: `/v3/projects/ba863e398e8a4c878aec56948cacddd2/users/14d562d970d04925a05f25f26b751771/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.816797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba863e398e8a4c878aec56948cacddd2, user_id=14d562d970d04925a05f25f26b751771, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:47.819510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:47.834591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1171/2345] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:11:47 2022] PUT /identity/v3/projects/ba863e398e8a4c878aec56948cacddd2/users/14d562d970d04925a05f25f26b751771/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:11:47.904707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:47.905118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:47.905406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:47.935347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] MFA Rules not processed for user `14d562d970d04925a05f25f26b751771`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:11:47.936591 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:47.999631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:11:48.004424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1175/2346] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:11:48.015253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:11:48.018390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:11:48.099197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4a43b-0a38-42f4-80b0-1349065db3c8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:11:48.103323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:11:48.103969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:11:48.104254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:11:48.104531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:11:48.113446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:11:48.114904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2801a1-d023-46f2-8685-d6a9802675f9 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:11:48.123962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1172/2347] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:11:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:11:58.119693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=50,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:00.480978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:00.482377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:00.539393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f528b8a-573e-4407-a181-b7dbde091c81 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:00.542908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:00.543544 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:00.543826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:00.544094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:00.545840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:00.619263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:00.620659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:00.630327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1176/2348] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:10.625956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:16.638542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5559-b3b3-470d-93d6-f6501cd93d50 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:16.639055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5559-b3b3-470d-93d6-f6501cd93d50 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:16.639155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b5559-b3b3-470d-93d6-f6501cd93d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:16.698429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-067b5559-b3b3-470d-93d6-f6501cd93d50 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:16.699698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-067b5559-b3b3-470d-93d6-f6501cd93d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:16.769396 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1173/2349] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:12:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:16.810499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:16.811970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:16.898892 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf6f28-dd30-45d7-9d42-47e08228a1f9 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:16.904825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:16.905490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:16.905750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:16.906034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:16.918903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:16.922240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36578f-e08a-4ff4-b5a0-0f1612e28672 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:16.932419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1177/2350] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:12:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:12:26.766539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=47 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:26.929191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=41 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:30.090295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:30.090911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:30.091563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:30.113993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] MFA Rules not processed for user `df44f27783d34370ad83470ffde558a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:30.116196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:30.182484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1174/2351] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:35.889296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:35.889788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:35.889888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:35.907486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:35.908695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:35.916590 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1178/2352] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:12:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:35.929735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:35.941796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ea86b-fb01-435b-882a-af503275fff5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:35.946935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:35.947510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:35.947818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:35.948094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:35.956617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:35.958718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425e0e-c005-40ad-b412-a9a852c6e41c service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:35.968158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1175/2353] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:12:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 07:12:37.482269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:37.547715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f594-380c-4623-841e-665ed6ea7c6e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:37.552829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:37.553681 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:37.553964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:37.554237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] PATH_INFO: `/v3/users/14d562d970d04925a05f25f26b751771` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:37.555440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d562d970d04925a05f25f26b751771)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:37.557341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:37.606039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d562d970d04925a05f25f26b751771'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:37.613425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d562d970d04925a05f25f26b751771'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:37.614039 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d562d970d04925a05f25f26b751771'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:37.628382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d562d970d04925a05f25f26b751771'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:37.629194 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d562d970d04925a05f25f26b751771'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:37.645000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1179/2354] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:12:37 2022] DELETE /identity/v3/users/14d562d970d04925a05f25f26b751771 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:37.649432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:37.649935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:37.650034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:37.670190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:37.671507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:37.735456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1176/2355] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:37.750539 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:37.753778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:37.810905 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a30c9-5be6-4255-87ba-067117d23d31 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:37.814470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:37.815156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:37.815436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:37.815710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:37.825242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:37.826667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:37.836757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1180/2356] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:37.974235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:37.975702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:37.989265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483b655-c1f2-49f4-b0cf-295646910e82 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:37.994674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:37.995745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:37.996032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:37.996311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] PATH_INFO: `/v3/users/6e8e19cb76464144a74a8504bcc42083` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:37.997550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8e19cb76464144a74a8504bcc42083)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:37.999522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:38.086807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8e19cb76464144a74a8504bcc42083'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.103689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8e19cb76464144a74a8504bcc42083'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.104869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8e19cb76464144a74a8504bcc42083'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.133533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8e19cb76464144a74a8504bcc42083'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.135383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8e19cb76464144a74a8504bcc42083'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.161236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1177/2357] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:12:37 2022] DELETE /identity/v3/users/6e8e19cb76464144a74a8504bcc42083 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:38.173189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:38.173879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:38.204717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9d2f7-bce5-46de-86c5-62f0665ce116 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:38.248029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:38.343622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:38.344870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.346035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.347028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] PATH_INFO: `/v3/projects/ba863e398e8a4c878aec56948cacddd2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.349103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba863e398e8a4c878aec56948cacddd2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:38.351626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:38.471865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba863e398e8a4c878aec56948cacddd2', 'request_id': 'req-f0df3beb-8153-429c-a849-720710b0e2dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.483372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:38.484890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba863e398e8a4c878aec56948cacddd2', 'request_id': 'req-f0df3beb-8153-429c-a849-720710b0e2dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.485358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:38.500551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba863e398e8a4c878aec56948cacddd2', 'request_id': 'req-f0df3beb-8153-429c-a849-720710b0e2dd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.505715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:38.514657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] The token cache is being invalidate because project ba863e398e8a4c878aec56948cacddd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:12:38.515086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acea035-2e9f-41dd-9ec2-61c27699c40c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:38.515386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.516423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0df3beb-8153-429c-a849-720710b0e2dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.519220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1181/2358] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:12:38 2022] DELETE /identity/v3/projects/ba863e398e8a4c878aec56948cacddd2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:38.524815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6cf1-0636-4b6b-9c33-5cc4d02310be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.525463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6cf1-0636-4b6b-9c33-5cc4d02310be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.527126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac6cf1-0636-4b6b-9c33-5cc4d02310be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.555426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dfac6cf1-0636-4b6b-9c33-5cc4d02310be None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:38.556695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dfac6cf1-0636-4b6b-9c33-5cc4d02310be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:38.622637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:38.624680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.625206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.627177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] PATH_INFO: `/v3/projects/1c5b07c9b6004015bc1e693324d246ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.629727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5b07c9b6004015bc1e693324d246ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:38.633892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:38.662826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1182/2359] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:38.668564 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.672571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.677903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.714739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5b07c9b6004015bc1e693324d246ce', 'request_id': 'req-c503b323-2b4d-4427-88f9-42655e433131'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.718577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:38.720004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:38.733489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5b07c9b6004015bc1e693324d246ce', 'request_id': 'req-c503b323-2b4d-4427-88f9-42655e433131'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.749586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5b07c9b6004015bc1e693324d246ce', 'request_id': 'req-c503b323-2b4d-4427-88f9-42655e433131'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.762931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] The token cache is being invalidate because project 1c5b07c9b6004015bc1e693324d246ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:12:38.763301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.763942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:38.765959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1178/2360] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:12:38 2022] DELETE /identity/v3/projects/1c5b07c9b6004015bc1e693324d246ce => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:38.789220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:38.818374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1183/2361] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:38.826530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:38.851681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503b323-2b4d-4427-88f9-42655e433131 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:38.858667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:38.860380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.860881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.861435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.862677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:12:38.863299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:38.866264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:38.883535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1179/2362] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:12:38.887503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:38.904721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260059-9a92-49a6-b7b6-8823f8d98f48 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:38.911111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:38.912538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:38.912891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:38.913237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:38.914167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:38.916404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:38.949814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1184/2363] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:12:38 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:12:38.954427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:38.970993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ec11f-f065-4f0f-9f01-4c4ff67a387f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.036929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.037666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.037955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.038231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.039251 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.041312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.065553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1180/2364] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:12:38 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:12:39.066474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.077223 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08841-176c-4cee-bf7e-903975199d8a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.082075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.082851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.083799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.084128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.084866 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.086483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.098365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1185/2365] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:12:39.103100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.113107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9372a26-e5d0-4ad0-ae9c-8891d97b7e8e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.118478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.119170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.119455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.119730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.120457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.122083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913a9e4-bce5-49a6-b6c4-4adaea628e24 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.127386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:39.134640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1181/2366] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:12:39.139548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.150949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f54ad5-5838-4ee7-a05b-d81cbec25b42 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.155673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.156553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.156833 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.157107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] PATH_INFO: `/v3/projects/b7f8b39c88ce44dc80d175725ea08435/users/4c5883f0ab4e446c86fade2c8b49a805/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.172395 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f8b39c88ce44dc80d175725ea08435, user_id=4c5883f0ab4e446c86fade2c8b49a805, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.175065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.189053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1186/2367] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:12:39 2022] PUT /identity/v3/projects/b7f8b39c88ce44dc80d175725ea08435/users/4c5883f0ab4e446c86fade2c8b49a805/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:39.200763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.201049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.201326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.229903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] MFA Rules not processed for user `4c5883f0ab4e446c86fade2c8b49a805`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:39.231191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:39.298220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1182/2368] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:39.310001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.311733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:39.370699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83df01-e497-4d9e-bad4-c61790808c0f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.374325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.374994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.375277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.375554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.383888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.385326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0527a3-eb05-41c7-8d9e-6e941d554b87 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.394177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1187/2369] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:39.418880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.482897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2829276-0dca-48d3-a773-b637efb71cc5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.487306 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.488160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.488445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.488717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] PATH_INFO: `/v3/users/4c5883f0ab4e446c86fade2c8b49a805` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.489947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5883f0ab4e446c86fade2c8b49a805)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.491886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.536888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5883f0ab4e446c86fade2c8b49a805'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:39.544087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5883f0ab4e446c86fade2c8b49a805'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:39.544760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5883f0ab4e446c86fade2c8b49a805'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:39.557169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5883f0ab4e446c86fade2c8b49a805'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:39.557819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5883f0ab4e446c86fade2c8b49a805'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:39.570391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1183/2370] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:12:39 2022] DELETE /identity/v3/users/4c5883f0ab4e446c86fade2c8b49a805 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:39.575520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.575996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.576116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.595431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:39.596637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:39.658510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1188/2371] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:39.668574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:39.669974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:39.681384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:39.731248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56dc04-6869-40dd-98aa-088fc669c922 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:39.735192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:39.735782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:39.736111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:39.736371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:39.748472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:39.750004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1b24b-f788-4a14-9667-77f6aebe0d3f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:39.759874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1184/2372] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:40.028126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.029631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:40.044691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e5f13-cca6-4b56-895b-8a400fde4279 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.049341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.050212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.050496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.050797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] PATH_INFO: `/v3/projects/b7f8b39c88ce44dc80d175725ea08435` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.052004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f8b39c88ce44dc80d175725ea08435)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.054595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.090183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:40.109179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f8b39c88ce44dc80d175725ea08435', 'request_id': 'req-59cbc23c-0ce8-48a5-9eea-631d935898d4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:40.116040 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f8b39c88ce44dc80d175725ea08435', 'request_id': 'req-59cbc23c-0ce8-48a5-9eea-631d935898d4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:40.123749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f8b39c88ce44dc80d175725ea08435', 'request_id': 'req-59cbc23c-0ce8-48a5-9eea-631d935898d4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:40.134518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] The token cache is being invalidate because project b7f8b39c88ce44dc80d175725ea08435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:12:40.134831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:40.135460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-59cbc23c-0ce8-48a5-9eea-631d935898d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:12:40.137021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1189/2373] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:12:40 2022] DELETE /identity/v3/projects/b7f8b39c88ce44dc80d175725ea08435 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:40.142144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.142434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.142755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.161030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:40.163641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:40.229501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1185/2374] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:40.237410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.237781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.238153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.257341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:40.258561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:40.266465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1190/2375] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:40.270936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.280885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae06c7-2899-48ae-a503-6aa39a127419 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.285793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.287064 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.287423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.287773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.288532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:12:40.288840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.290483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.303009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1186/2376] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:12:40.307216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.317020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d0ba4-0037-4568-8f48-5abdeabc8890 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.322266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.323225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.323225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.323490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.324168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.326067 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.355228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1191/2377] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:12:40 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:12:40.356354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.367117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55175a9-4961-4fb2-ade0-5fb3f52de6ac admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.423495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.424165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.424449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.424729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.425423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.427881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.449360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1187/2378] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:12:40 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:12:40.453251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.463863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375747a6-7665-4ead-adf8-ad15cd6e793c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.468939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.469726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.470011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.470285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.471048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.472665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.484420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1192/2379] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:12:40.488800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.504355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c473c-74ed-41ed-a8d3-ad16ae89232f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.511683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.512607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.512984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.513677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.514171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.516607 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5276c5-196a-4b1b-b6f4-44965ed75c75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.530160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1188/2380] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:12:40.534160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.546853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57762128-c899-4756-b103-5ac5399e8c72 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.554378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.555484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.555875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.556260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] PATH_INFO: `/v3/projects/ce460f857d824071927f15e19cbb11e7/users/631f377cb70b454cbde19de83676f502/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.576350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce460f857d824071927f15e19cbb11e7, user_id=631f377cb70b454cbde19de83676f502, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.579113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.596052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1193/2381] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:12:40 2022] PUT /identity/v3/projects/ce460f857d824071927f15e19cbb11e7/users/631f377cb70b454cbde19de83676f502/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:40.601368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05527f-c5f1-4394-9aaa-ec467ffeea28 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.601665 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05527f-c5f1-4394-9aaa-ec467ffeea28 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.601951 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05527f-c5f1-4394-9aaa-ec467ffeea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.620873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bc05527f-c5f1-4394-9aaa-ec467ffeea28 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:40.622096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc05527f-c5f1-4394-9aaa-ec467ffeea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:40.635219 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:40.691761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1189/2382] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:40.701673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:40.703371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:40.758749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85901a-84b8-45c7-8600-f34be6b4d01f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:40.762319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:40.762985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:40.763266 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:40.763542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:40.772398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:40.773763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:40.783012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1194/2383] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:45.343739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.344340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.344340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.374543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] MFA Rules not processed for user `631f377cb70b454cbde19de83676f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:45.375899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:45.476689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1190/2384] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:45.477745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.487970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df65db-f97d-405a-ac8b-1fc4a65f5548 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.493433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.494215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.494582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.494958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.495754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.497858 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.536588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1195/2385] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:12:45.540527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.558519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528607f-f58b-4071-afb4-a001dac666aa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.625027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.625305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.625592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.625866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.626574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.628553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.662235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1191/2386] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:12:45 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:12:45.663183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.675097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7511-ae52-45c5-9811-05eb3e6fa604 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.680753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.681420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.681699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.681973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.682730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.684751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.696697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1196/2387] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:12:45.700786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.711254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b688-1039-4ef4-8ae8-e738ba3c6d66 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.716138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.717031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.717312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.717586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] PATH_INFO: `/v3/projects/3faf9aa722c24908ab91db2d7449e758/users/741a67a1b8b3407bbc56aa695bb9c349/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.733733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3faf9aa722c24908ab91db2d7449e758, user_id=741a67a1b8b3407bbc56aa695bb9c349, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.736686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.751359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1192/2388] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:12:45 2022] PUT /identity/v3/projects/3faf9aa722c24908ab91db2d7449e758/users/741a67a1b8b3407bbc56aa695bb9c349/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:45.756199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.767452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97f366-b2de-4f0a-8b48-b834751c3a18 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.785912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:45.829697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.830284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.830569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.830877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.831607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.833279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac57d31-346c-4a3d-94fd-091daa7db532 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.847154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1197/2389] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:12:45.849109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:45.863750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85213230-6a36-4b2b-8761-88d329044379 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:45.869410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:45.870309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:45.870648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:45.870981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] PATH_INFO: `/v3/projects/3faf9aa722c24908ab91db2d7449e758/users/741a67a1b8b3407bbc56aa695bb9c349/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:45.873049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3faf9aa722c24908ab91db2d7449e758, user_id=741a67a1b8b3407bbc56aa695bb9c349, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:45.875741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:45.883615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:12:45.891911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1193/2390] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:12:45 2022] PUT /identity/v3/projects/3faf9aa722c24908ab91db2d7449e758/users/741a67a1b8b3407bbc56aa695bb9c349/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:12:48.135902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca1dc-fdb7-4b6d-bed7-1534919e21e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:48.136431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca1dc-fdb7-4b6d-bed7-1534919e21e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:48.136727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca1dc-fdb7-4b6d-bed7-1534919e21e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:48.161906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f6dca1dc-fdb7-4b6d-bed7-1534919e21e0 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:48.163369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f6dca1dc-fdb7-4b6d-bed7-1534919e21e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:48.276687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1198/2391] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:12:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:48.289529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:48.291767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:48.401047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417dfed-2630-43c8-b4b1-3f99c61cc0c2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:48.408903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:48.409732 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:48.410019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:48.410344 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:48.420692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:48.422301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:48.435168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1194/2392] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:12:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 07:12:52.830126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2942-bd4d-4464-b11b-dbe0d0ca391f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:52.830728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2942-bd4d-4464-b11b-dbe0d0ca391f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:52.830728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2942-bd4d-4464-b11b-dbe0d0ca391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:52.877421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6bdf2942-bd4d-4464-b11b-dbe0d0ca391f None None] MFA Rules not processed for user `741a67a1b8b3407bbc56aa695bb9c349`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:52.879186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6bdf2942-bd4d-4464-b11b-dbe0d0ca391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:53.002401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1199/2393] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:12:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:53.023171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:53.027039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:53.114983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264747bc-3eac-4170-be97-845b7f42f901 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:53.119247 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:53.120480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:53.121033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:53.121671 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:53.207018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:53.209439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:53.221128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1195/2394] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:12:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:12:53.382035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f5db-c02c-4f8d-8b21-c6f47ac97bfd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:53.382489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f5db-c02c-4f8d-8b21-c6f47ac97bfd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:53.382839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5f5db-c02c-4f8d-8b21-c6f47ac97bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:53.408868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-31c5f5db-c02c-4f8d-8b21-c6f47ac97bfd None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:12:53.410414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-31c5f5db-c02c-4f8d-8b21-c6f47ac97bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:12:53.420374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1200/2395] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:12:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:12:53.439092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:12:53.453803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d889b-8f8e-4f1d-b05c-34731fad7f5a service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:12:53.460284 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:12:53.465036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:12:53.465337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:12:53.465597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:12:53.480053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:12:53.482044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3c413-61c2-4d8f-bf70-1042726a07ab service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:12:53.493739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1196/2396] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:12:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:13:03.399623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:03.489545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:04.561312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:04.561784 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:04.561899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:04.578926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] MFA Rules not processed for user `631f377cb70b454cbde19de83676f502`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:04.580115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:04.587698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1201/2397] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:13:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:13:09.293541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:09.294042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:09.294140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:09.313361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:09.314577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:09.322443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1197/2398] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:13:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:13:09.334981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:09.347059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8a5b-6533-4056-8945-4d926ff3dd3d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:09.351937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:09.352579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:09.352865 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:09.353148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:09.361779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:09.363730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:09.373037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1202/2399] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:13:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:13:15.752472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:15.809352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa132a-8541-4f21-8990-dd40bec6c935 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:15.812956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:15.813615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:15.813890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:15.814166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:15.816054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:15.904684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:15.906131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:15.926709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1198/2400] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:17.926732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:17.936842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e60fdf-e8b6-4f9e-a53a-a8a9a8e521c0 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:17.940806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:17.942044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:17.942339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:17.942616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:17.944582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:17.958587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:17.960054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ca806-c89a-45af-9fcf-6c401f17e008 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:17.971144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1203/2401] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:25.923428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=66 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:27.966000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=15 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:29.827682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:29.837004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6cd90-4b3a-439b-beaf-a04dbf6990cb service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:29.840847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:29.841498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:29.841785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:29.842066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:29.843920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:29.853773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:29.855232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9829d-5622-424b-8308-32114ffccb52 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:29.863265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1199/2402] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:29.970982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:29.971921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:29.972415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:29.974350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1204/2403] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:13:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:13:29.981422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174669e5-e9f4-4e20-8400-715901e07de6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:29.981881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174669e5-e9f4-4e20-8400-715901e07de6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:29.982311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174669e5-e9f4-4e20-8400-715901e07de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.023567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-174669e5-e9f4-4e20-8400-715901e07de6 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:30.024822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-174669e5-e9f4-4e20-8400-715901e07de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.034972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1200/2404] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:13:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:13:30.046821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:30.048604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.063003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c13f-cdf3-4f70-88f7-ef6dc4f7ac35 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:30.067524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:30.068164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.068446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.068751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.079083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:30.080507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142591e-47ac-4ac4-adc8-e9409c7c3b4d service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:30.091482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1205/2405] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:30.164458 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.164751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.165041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.166454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1201/2406] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:13:30.171503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112257-0b5a-4e80-a719-95d578fb7883 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.171864 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112257-0b5a-4e80-a719-95d578fb7883 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.172210 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74112257-0b5a-4e80-a719-95d578fb7883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.191400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-74112257-0b5a-4e80-a719-95d578fb7883 None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:30.192617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-74112257-0b5a-4e80-a719-95d578fb7883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.199814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1206/2407] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:13:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:13:30.212233 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:30.214076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.228314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde9ab-f7f7-42a0-8593-59a2ed106db6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:30.232077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:30.232823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.233178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.233551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.241589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:30.243185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364f879-7eda-4b7a-9154-49be4bb1a242 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:30.251768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1202/2408] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:30.334496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa62-63f8-4248-b1e9-0250e73423d1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.334808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa62-63f8-4248-b1e9-0250e73423d1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.335096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa62-63f8-4248-b1e9-0250e73423d1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.336478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1207/2409] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:13:30.341809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.342152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.342432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.360930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:30.362057 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.371946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1203/2410] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:13:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:13:30.382985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398fa62-63f8-4248-b1e9-0250e73423d1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:30.393873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398fa62-63f8-4248-b1e9-0250e73423d1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:30.399463 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:30.400227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.400514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.400787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.411751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:30.413179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:30.424685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1208/2411] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:30.487657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:30.489120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.555925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26f628-b23b-4a3a-8c1b-70dc9dfcdf07 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:30.561254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:30.561898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:30.562180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:30.562454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:30.564261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:30.575957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:30.577532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:30.588357 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1204/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:31.317553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:31.319909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:31.326813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:31.328585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:31.337442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8273d-bf75-495f-a188-94e65a2e1c5e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:31.344117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:31.344876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:31.345226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:31.345761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:31.348875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:31.407938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc9e92-7800-491a-9002-005d96d7f55a service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:31.412899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:31.413662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:31.414059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:31.414417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:31.416770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:31.479233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:31.480085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:31.481549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:31.493341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1209/2413] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:31.493653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:31.495114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52f955-fe39-4033-9783-ed94f71135b4 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:31.504812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1205/2414] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:38.803160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:38.812876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec602d5-dd2d-411a-8b55-17154eb42153 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:38.817068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:38.817703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:38.817983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:38.818260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:38.820122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:38.831065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:38.833086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:38.841703 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1210/2415] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:39.604871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:39.605371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:39.605472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:39.624931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:39.626067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:39.633361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1206/2416] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:13:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:13:39.654853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:39.667182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c9daf-99c8-4b06-9512-5cf143506b89 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:39.672259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:39.674173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:39.674173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:39.674173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:39.683690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:39.685607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463fe8d-92f4-47a6-a327-c5fb18df32fd service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:39.696307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1211/2417] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:13:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:13:40.834925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:13:40.836394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:40.852378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4696f4-6cab-4dcb-ac29-e009b264ee02 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:13:40.857868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:13:40.858503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:40.858811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:40.859093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:40.860829 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:40.925817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:13:40.927553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:13:40.938166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1207/2418] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:13:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:13:48.530880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac467-40e0-4a3d-af12-c8516079b41f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:13:48.531454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac467-40e0-4a3d-af12-c8516079b41f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:13:48.531454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cac467-40e0-4a3d-af12-c8516079b41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:13:48.551621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c5cac467-40e0-4a3d-af12-c8516079b41f None None] MFA Rules not processed for user `1957cb1501da480f9974bda730ce9db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:13:48.552814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c5cac467-40e0-4a3d-af12-c8516079b41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:13:48.608781 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1212/2419] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:13:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:13:50.932793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=35 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:13:58.601358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=80 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:14:02.261898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:14:02.278276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6283da-adc8-4234-827c-dffc80fca252 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:14:02.284549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:14:02.285319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:02.285598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:02.285875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:02.287739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:14:02.304859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:14:02.306726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:14:02.330441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1208/2420] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:14:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:14:12.313879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:14:13.816663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:13.817143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:13.817241 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:13.834420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:14:13.835685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:14:13.842733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1213/2421] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:14:13.854928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:14:13.866690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905b9de-4406-440f-bd05-8296236a61b2 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:14:13.871743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:14:13.872375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:13.872660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:13.872956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:13.880918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:14:13.882525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18c704-3410-4c80-a190-cabfd78ccc7b service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:14:13.890508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1209/2422] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:14:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:14:17.526559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:14:17.536101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa1b36-9e61-4db7-82d9-6101f6bdc3c7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:14:17.539646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:14:17.540283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:17.541146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:17.541430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:17.550325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:14:17.551790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:14:17.561398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1214/2423] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:14:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:14:23.887329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:14:27.557059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=9 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:14:36.234718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35df0b-7743-48b0-907b-d440a1a69468 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:36.235211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35df0b-7743-48b0-907b-d440a1a69468 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:36.235309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35df0b-7743-48b0-907b-d440a1a69468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:36.252509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7b35df0b-7743-48b0-907b-d440a1a69468 None None] MFA Rules not processed for user `4549c33736bc413a87460db0c43f5768`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:14:36.254202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7b35df0b-7743-48b0-907b-d440a1a69468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:14:36.261167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1210/2424] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:14:46.247264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:14:56.779474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:14:56.792190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4752506-1c1f-4c22-8168-8c05e2427166 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:14:56.797525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:14:56.798151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:14:56.798432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:14:56.798731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:14:56.801075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:14:56.856721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:14:56.858290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:14:56.868269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1215/2425] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:14:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:06.864125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=28 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:15:15.136285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:15.136785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:15.136897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:15.154940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] MFA Rules not processed for user `df44f27783d34370ad83470ffde558a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:15:15.156286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:15.163341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1211/2426] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:15:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:15:18.903391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:18.916213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71227c3-db40-4e80-bbb9-15f6e11e8c41 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:18.921457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:18.922333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:18.922791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:18.923164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:18.925081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:18.939660 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:18.943264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:18.955053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1216/2427] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:18.962941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:18.964348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:19.042948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9f461-c28b-45b5-9b10-c2e4dca3fd54 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:19.047582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:19.048445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:19.048740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:19.048999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] PATH_INFO: `/v3/users/1957cb1501da480f9974bda730ce9db3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:19.050314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1957cb1501da480f9974bda730ce9db3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:19.052262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:19.101845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1957cb1501da480f9974bda730ce9db3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.112409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1957cb1501da480f9974bda730ce9db3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.112997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1957cb1501da480f9974bda730ce9db3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.133320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1957cb1501da480f9974bda730ce9db3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.134062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1957cb1501da480f9974bda730ce9db3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.147457 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1212/2428] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:15:18 2022] DELETE /identity/v3/users/1957cb1501da480f9974bda730ce9db3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:15:19.407283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:19.417715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687bb6-3d06-4357-ae21-1eed8be51aac service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:19.478788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:19.479670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:19.479955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:19.480230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] PATH_INFO: `/v3/projects/6dd3241e9f134a7bb89bbaf40232959c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:19.481390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd3241e9f134a7bb89bbaf40232959c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:19.483339 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:19.529923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd3241e9f134a7bb89bbaf40232959c', 'request_id': 'req-b673f780-5856-458e-b201-031210d8b024'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.535937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd3241e9f134a7bb89bbaf40232959c', 'request_id': 'req-b673f780-5856-458e-b201-031210d8b024'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.543088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd3241e9f134a7bb89bbaf40232959c', 'request_id': 'req-b673f780-5856-458e-b201-031210d8b024'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.553548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] The token cache is being invalidate because project 6dd3241e9f134a7bb89bbaf40232959c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:15:19.553831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.554454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:15:19.555986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1217/2429] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:15:19 2022] DELETE /identity/v3/projects/6dd3241e9f134a7bb89bbaf40232959c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:15:23.187411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:23.190985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:23.267272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21621217-e5e4-452e-9341-1cb608b71b6d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:23.276076 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:15:23.338267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:23.338961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:23.339248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:23.339535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:23.341320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:23.353291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:23.354977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08741c0-77d7-490b-935c-dc73dcba9e7c service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:23.367991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1213/2430] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:29.550636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:15:29.755884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:29.757322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:29.813073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673f780-5856-458e-b201-031210d8b024 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:29.818444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:29.819609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:29.819995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:29.820279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:29.823489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:29.839276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:29.840713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea595cc-8f2a-477d-a4d5-4766e4f7d30c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:29.848583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1218/2431] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:29.949109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:29.949668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:29.949668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:29.951196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1214/2432] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:15:29.957587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4e05-56ab-40c2-93d1-182b37fd5274 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:29.957958 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4e05-56ab-40c2-93d1-182b37fd5274 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:29.958303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce4e05-56ab-40c2-93d1-182b37fd5274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:29.976896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-24ce4e05-56ab-40c2-93d1-182b37fd5274 None None] MFA Rules not processed for user `58947ba37ab14f61adde0bf2f34829f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:15:29.978218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-24ce4e05-56ab-40c2-93d1-182b37fd5274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:29.985068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1219/2433] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:15:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:15:29.998007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:29.999730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:30.013234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956d77-52a8-4610-bdd7-a380e23a0316 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:30.016726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:30.017383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:30.017667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:30.017942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:30.025170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:30.026572 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71158944-8eff-4bd0-8ccf-b8c6793ad36e service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:30.034920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1215/2434] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:30.116618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be839-8c6a-45d9-bfad-096437efe5aa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:30.116909 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be839-8c6a-45d9-bfad-096437efe5aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:30.117189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082be839-8c6a-45d9-bfad-096437efe5aa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:30.118613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1220/2435] 10.222.0.48 () {54 vars in 921 bytes} [Fri Jul 1 07:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 01 07:15:30.123468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:30.123756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:30.124037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:30.149838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] MFA Rules not processed for user `458dc4b52e3d4a13a8a4f801ddf2e6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:15:30.151078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:30.160327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1216/2436] 10.222.0.48 () {58 vars in 1018 bytes} [Fri Jul 1 07:15:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:15:30.170647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082be839-8c6a-45d9-bfad-096437efe5aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:30.182191 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082be839-8c6a-45d9-bfad-096437efe5aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:30.186022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:30.186652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:30.186995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:30.187273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:30.198418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:30.199894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0beb44c-92b7-47d4-8a8d-4d613f5f8d84 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:30.210595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1221/2437] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:40.141983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=38 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:15:40.205925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=35 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:15:45.167682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:15:45.169090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:45.182896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0824-1aba-4767-9501-c69c62339014 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:15:45.188933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:15:45.189590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:45.189877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:45.190159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:45.191967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:45.249766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:15:45.251969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:15:45.261470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1217/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:15:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:15:50.702869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:15:50.703371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:15:50.703477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:15:50.720963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] MFA Rules not processed for user `0e8bf7bbabc4432fa88002ea5c1dce4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:15:50.722207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:15:50.776327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1222/2439] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:15:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:15:55.256927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=28 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:00.769617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:12.045141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:12.054575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6e106-b260-471e-b673-ef2e36b0845b service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:12.058116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:12.058747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:12.059350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:12.059636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:12.061376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:12.121960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:12.123479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:12.133205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1218/2440] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:15.370013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:15.371568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:15.436257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:15.438296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:15.451420 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99cab9-dda6-43e1-9fbf-7289f42c2ca7 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:15.454124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f4f54-abd2-4079-b58b-a4a4f8189fee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:15.460972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:15.462108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:15.462418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:15.462695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] PATH_INFO: `/v3/users/4549c33736bc413a87460db0c43f5768` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:15.463928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4549c33736bc413a87460db0c43f5768)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:15.464516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:15.465252 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:15.465611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:15.465856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:15.465967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:15.468631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:15.481873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:15.484014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:15.496797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1223/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:15.521398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4549c33736bc413a87460db0c43f5768'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.533215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4549c33736bc413a87460db0c43f5768'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.534089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4549c33736bc413a87460db0c43f5768'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.550484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4549c33736bc413a87460db0c43f5768'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.551235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8ee4350-d886-48f2-a86e-8bff85e68f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4549c33736bc413a87460db0c43f5768'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.565684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1219/2442] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:15 2022] DELETE /identity/v3/users/4549c33736bc413a87460db0c43f5768 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:15.744087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:15.755604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6f5fa-b3b4-4b0f-a9bf-619d386d6ed0 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:15.812570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:15.814035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:15.814320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:15.814593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] PATH_INFO: `/v3/projects/d94c1c234804421487c49cc9df6ff888` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:15.815988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94c1c234804421487c49cc9df6ff888)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:15.818179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:15.865945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94c1c234804421487c49cc9df6ff888', 'request_id': 'req-083bb6f0-42e4-4575-a7b1-61c14b3276ff'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.871972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94c1c234804421487c49cc9df6ff888', 'request_id': 'req-083bb6f0-42e4-4575-a7b1-61c14b3276ff'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.879514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94c1c234804421487c49cc9df6ff888', 'request_id': 'req-083bb6f0-42e4-4575-a7b1-61c14b3276ff'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.889455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] The token cache is being invalidate because project d94c1c234804421487c49cc9df6ff888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:15.889740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.890357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-083bb6f0-42e4-4575-a7b1-61c14b3276ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:15.891963 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1224/2443] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:15 2022] DELETE /identity/v3/projects/d94c1c234804421487c49cc9df6ff888 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:15.896777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:15.897072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:15.897355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:15.916304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:15.917503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:15.975349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1220/2444] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:15.983510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:15.983802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:15.984088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.002841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:16.004028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:16.011722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1225/2445] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:16.015626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.030531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bcb2-b0dd-460d-bb70-ec12c1d26259 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.035647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.036302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.036584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.036859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.037584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:16.037890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.039525 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.040953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:16.056246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1221/2446] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:16:16.057424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.066908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3e2d5-96eb-40d7-b562-c3a7605d5d52 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.071523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.072162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.072448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.072721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.073397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.075333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.084616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:16.111942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1226/2447] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:16.116144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.126140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487de4c-5513-4965-ad9d-9e393e51a08c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.196144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.196904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.197254 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.197597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.198314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.200245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.226524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1222/2448] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:16.227816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.238822 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c654a-05e3-4abc-a75e-4d35338b9940 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.243430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.244085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.244366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.244637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.245374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.247031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.257213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1227/2449] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:16.261158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.271435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d95b9-0620-4e33-95fa-f8657b28c7b4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.276041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.276697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.276974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.277246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.277966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.279651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a949e62-7471-45cf-8d15-649d8181b7cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.290661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1223/2450] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:16:16.294472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.305052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8b6f6-19b1-4755-a377-13b223985341 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.310391 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.311880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.312170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.312449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] PATH_INFO: `/v3/projects/d2f9786aab774cfeb09d318d3998996b/users/818012c25f9048fdbb4edaecce3ea7b8/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.327595 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f9786aab774cfeb09d318d3998996b, user_id=818012c25f9048fdbb4edaecce3ea7b8, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.330322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.344712 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1228/2451] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:16 2022] PUT /identity/v3/projects/d2f9786aab774cfeb09d318d3998996b/users/818012c25f9048fdbb4edaecce3ea7b8/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:16.354563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.354886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.355166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.381225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] MFA Rules not processed for user `818012c25f9048fdbb4edaecce3ea7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:16.382350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:16.444791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1224/2452] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:16:16.449087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.512474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe998c-61a4-495c-ba71-f3e39a313490 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.517675 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.518331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.518616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.518934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.519630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.521552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.546639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1229/2453] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:16.550746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.561584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb51a3-35db-490e-9d2d-4c9e5e034053 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.617942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.618626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.618943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.619221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.619938 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.622132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.630563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:16.644791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1225/2454] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:16.648616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.659800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c393d-c7f3-4431-a12a-319b816b70c3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.664288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.664927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.665208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.665480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.666189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.667801 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.677986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1230/2455] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:16.681717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.691991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf94ac-ee3a-413f-8618-e32da1af7432 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.696577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.697442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.697728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.698005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] PATH_INFO: `/v3/projects/4c3c21819fc342988f27cada8ca3e919/users/26a1817f9cf6456a8334eb4314dd0b43/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.712841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3c21819fc342988f27cada8ca3e919, user_id=26a1817f9cf6456a8334eb4314dd0b43, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.715812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.730213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1226/2456] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:16 2022] PUT /identity/v3/projects/4c3c21819fc342988f27cada8ca3e919/users/26a1817f9cf6456a8334eb4314dd0b43/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:16.734151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.744025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bca25-b264-49d7-b0c9-8e218a9f6879 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.800439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.801098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.801381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.801657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.802369 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.804027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0c1e6-a5fd-4c50-8792-9c708b885490 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.814718 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1231/2457] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:16.819395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.830864 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a3945-2e09-4318-a70f-bfbcb5d679ec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:16.835468 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:16.836340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.836616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.836885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] PATH_INFO: `/v3/projects/4c3c21819fc342988f27cada8ca3e919/users/26a1817f9cf6456a8334eb4314dd0b43/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.838851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3c21819fc342988f27cada8ca3e919, user_id=26a1817f9cf6456a8334eb4314dd0b43, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:16.841584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:16.851617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1227/2458] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:16 2022] PUT /identity/v3/projects/4c3c21819fc342988f27cada8ca3e919/users/26a1817f9cf6456a8334eb4314dd0b43/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:16.861228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:16.861533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:16.861842 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:16.888122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] MFA Rules not processed for user `26a1817f9cf6456a8334eb4314dd0b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:16.889327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:16.894734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:16.957348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1232/2459] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:16.968569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:16.970312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:17.026205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fab728-45d9-4662-a0b7-b66355b87f73 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:17.030130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:17.030762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:17.031082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:17.031361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:17.040417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:17.041890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873900a-a6c0-4cdc-8a81-92f9c54c2d42 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:17.051789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1228/2460] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:17.426011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:17.491347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e2082-3c77-405c-9795-36f1f7fd1383 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:17.495948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:17.496870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:17.497153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:17.497474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] PATH_INFO: `/v3/users/818012c25f9048fdbb4edaecce3ea7b8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:17.498665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818012c25f9048fdbb4edaecce3ea7b8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:17.500663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:17.555891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818012c25f9048fdbb4edaecce3ea7b8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:17.563117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818012c25f9048fdbb4edaecce3ea7b8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:17.563752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818012c25f9048fdbb4edaecce3ea7b8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:17.575496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818012c25f9048fdbb4edaecce3ea7b8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:17.576127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818012c25f9048fdbb4edaecce3ea7b8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:17.588095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1233/2461] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:17 2022] DELETE /identity/v3/users/818012c25f9048fdbb4edaecce3ea7b8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:17.592728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:17.593185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:17.593295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:17.611823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:17.613022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:17.670177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1229/2462] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:16:17.680545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:17.681937 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:17.707918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:17.742441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0500c67-36fd-42ad-adc5-1258f05da7c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:17.746051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:17.746691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:17.747004 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:17.747279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:17.756250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:17.757679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:17.767029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1234/2463] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:18.027022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.028897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:18.058209 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6db2-b5d6-467a-b8e1-374d755de1f0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:18.063065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:18.063949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.064229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.064502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] PATH_INFO: `/v3/projects/d2f9786aab774cfeb09d318d3998996b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.065716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f9786aab774cfeb09d318d3998996b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:18.067656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:18.113822 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f9786aab774cfeb09d318d3998996b', 'request_id': 'req-a5a4380d-97c7-4ea9-b69b-1def8d45a192'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.116413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:18.121803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f9786aab774cfeb09d318d3998996b', 'request_id': 'req-a5a4380d-97c7-4ea9-b69b-1def8d45a192'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.129405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f9786aab774cfeb09d318d3998996b', 'request_id': 'req-a5a4380d-97c7-4ea9-b69b-1def8d45a192'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.139609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] The token cache is being invalidate because project d2f9786aab774cfeb09d318d3998996b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:18.139925 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.141369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.142937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1230/2464] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:18 2022] DELETE /identity/v3/projects/d2f9786aab774cfeb09d318d3998996b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:18.146988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.156431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbd69-8d02-4e4a-b08d-21a1fe992e7f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:18.211349 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:18.212219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.212499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.212795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] PATH_INFO: `/v3/users/26a1817f9cf6456a8334eb4314dd0b43` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.214156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a1817f9cf6456a8334eb4314dd0b43)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:18.216092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:18.260777 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a1817f9cf6456a8334eb4314dd0b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.268344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a1817f9cf6456a8334eb4314dd0b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.269115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a1817f9cf6456a8334eb4314dd0b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.282000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a1817f9cf6456a8334eb4314dd0b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.282129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7855e9ab-2037-45e3-b536-b4fdb1d5d663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a1817f9cf6456a8334eb4314dd0b43'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.293914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1235/2465] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:18 2022] DELETE /identity/v3/users/26a1817f9cf6456a8334eb4314dd0b43 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:18.560445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.561936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:18.579650 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4380d-97c7-4ea9-b69b-1def8d45a192 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:18.642608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:18.643523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.643966 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.644272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] PATH_INFO: `/v3/projects/4c3c21819fc342988f27cada8ca3e919` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.646164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3c21819fc342988f27cada8ca3e919)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:18.647756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:18.697286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3c21819fc342988f27cada8ca3e919', 'request_id': 'req-eaf511fc-d81d-4eee-9d11-5e47cac4745f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.706099 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3c21819fc342988f27cada8ca3e919', 'request_id': 'req-eaf511fc-d81d-4eee-9d11-5e47cac4745f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.712894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3c21819fc342988f27cada8ca3e919', 'request_id': 'req-eaf511fc-d81d-4eee-9d11-5e47cac4745f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.725212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] The token cache is being invalidate because project 4c3c21819fc342988f27cada8ca3e919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:18.725507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.726178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eaf511fc-d81d-4eee-9d11-5e47cac4745f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:18.727835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1231/2466] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:18 2022] DELETE /identity/v3/projects/4c3c21819fc342988f27cada8ca3e919 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:18.732738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.733209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.733342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.757451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:18.758665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:18.798573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:18.820224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1236/2467] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:18.829016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.829389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.829733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.848010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:18.849220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:18.857019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1232/2468] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:18.861586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.871328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974c9ae-b6d1-4777-bc50-232682a929a1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:18.876569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:18.877220 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.877507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.877792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.878520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:18.878870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:18.880477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:18.891934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1237/2469] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:16:18.895687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.905096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ee131-f1c6-4acc-bdd7-cf32236bb648 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:18.909698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:18.910331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:18.910615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:18.910926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:18.911604 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:18.913479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:18.939075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1233/2470] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:18 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:18.943088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:18.952896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b458-8981-4856-8aa4-4348f469a2bf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.008601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.009258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.009540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.009818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.010503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.012489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.031405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1238/2471] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:18 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:19.036116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.048476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f37baf-7977-4540-9731-bc5b6f544dde admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.053189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.053840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.054119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.054389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.055152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.056725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.067237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1234/2472] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:16:19.071203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.080972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369ea3d-6d90-4560-918b-f7c1f571ad79 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.085556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.086203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.086488 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.086791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.087522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.089104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d6b2a-f12b-4573-8cf3-10159764eeef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.099416 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1239/2473] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:19.103024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.113232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01fa1a-46f4-46bd-98fd-77d96e1f48ec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.117901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.118781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.119095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.119369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] PATH_INFO: `/v3/projects/5fbd2c2b3b6f418da7e06ad768ea5d37/users/cc618a286b574fd1a0bf698680b9af72/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.133577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbd2c2b3b6f418da7e06ad768ea5d37, user_id=cc618a286b574fd1a0bf698680b9af72, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.136264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.148488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1235/2474] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:19 2022] PUT /identity/v3/projects/5fbd2c2b3b6f418da7e06ad768ea5d37/users/cc618a286b574fd1a0bf698680b9af72/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:19.158898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.159188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.159462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.185947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] MFA Rules not processed for user `cc618a286b574fd1a0bf698680b9af72`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:19.187236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:19.250322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1240/2475] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:19.254973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.263668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:19.316751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805244-2da5-43cc-887b-823b5d088437 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.321446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.322029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.322312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.322589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.323308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.325237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.350728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1236/2476] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:19 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:19.355096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.366306 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c9439-7d84-45b3-95b7-6dd0774bd11a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.403081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:19.422907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.423579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.423868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.424150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.424838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.426797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.446165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1241/2477] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:19 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:19.450098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.461548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d561930-817c-434a-bf98-9b5915ca177b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.466246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.466952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.467241 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.467523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.468281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.470541 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.481132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1237/2478] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:19.485397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.495318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8c86-2f14-48ed-9b74-a5712d2c74fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.499896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.500768 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.501044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.501314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] PATH_INFO: `/v3/projects/0b95465d3800437d816572fb656576ff/users/4f9a5cc076d644589a02f7b489fe278e/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.516630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b95465d3800437d816572fb656576ff, user_id=4f9a5cc076d644589a02f7b489fe278e, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.519680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.532601 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1242/2479] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:19 2022] PUT /identity/v3/projects/0b95465d3800437d816572fb656576ff/users/4f9a5cc076d644589a02f7b489fe278e/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:19.536669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.546660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10868-09d8-4dfa-bf2b-4699784916ce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.602409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.603097 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.603381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.603657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.604370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.605963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bba31-9693-4224-9319-4c7feb6b82e3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.615463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1238/2480] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:19.619614 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.629710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe847a-4b2f-462e-a3d0-b0168564d41c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.634294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.635205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.635489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.635766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] PATH_INFO: `/v3/projects/0b95465d3800437d816572fb656576ff/users/4f9a5cc076d644589a02f7b489fe278e/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.637713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b95465d3800437d816572fb656576ff, user_id=4f9a5cc076d644589a02f7b489fe278e, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.640397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.650885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1243/2481] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:19 2022] PUT /identity/v3/projects/0b95465d3800437d816572fb656576ff/users/4f9a5cc076d644589a02f7b489fe278e/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:19.662587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.663098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.663191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.689068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] MFA Rules not processed for user `4f9a5cc076d644589a02f7b489fe278e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:19.690268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:19.758296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1239/2482] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:19.770546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:19.772296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:19.828256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a5cc2-24e2-4bef-a6a0-fe073af1d36e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:19.832555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:19.833198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:19.833483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:19.833759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:19.842705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:19.844161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b903f-ea35-42b8-aa03-3f8c58e88d5d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:19.853505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1244/2483] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:20.245017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:20.249324 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:20.306216 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40fda2-9bd1-4da6-ab08-b33d0d734bb3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:20.310976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:20.311827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:20.312106 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:20.312379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] PATH_INFO: `/v3/users/cc618a286b574fd1a0bf698680b9af72` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:20.314181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc618a286b574fd1a0bf698680b9af72)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:20.316117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:20.360765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc618a286b574fd1a0bf698680b9af72'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:20.369073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc618a286b574fd1a0bf698680b9af72'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:20.369701 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc618a286b574fd1a0bf698680b9af72'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:20.382265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc618a286b574fd1a0bf698680b9af72'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:20.382926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc618a286b574fd1a0bf698680b9af72'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:20.394678 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1240/2484] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:20 2022] DELETE /identity/v3/users/cc618a286b574fd1a0bf698680b9af72 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:20.399211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:20.399506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:20.399792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:20.420107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:20.421337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:20.483803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1245/2485] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:20.494337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:20.495786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:20.551975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204f29d1-43ff-4e34-9a61-b69f3962cea1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:20.555633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:20.556271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:20.556557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:20.556838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:20.566135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:20.567590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:20.576996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1241/2486] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:21.033947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.035432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:21.048624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a476f-928d-4707-94ed-93a8383fa542 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.053409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.054277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.054560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.054941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] PATH_INFO: `/v3/projects/5fbd2c2b3b6f418da7e06ad768ea5d37` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.056049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbd2c2b3b6f418da7e06ad768ea5d37)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.057964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:21.091746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:21.106310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbd2c2b3b6f418da7e06ad768ea5d37', 'request_id': 'req-c48ab82a-1110-4f32-894a-6d2f65bd0148'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.112414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbd2c2b3b6f418da7e06ad768ea5d37', 'request_id': 'req-c48ab82a-1110-4f32-894a-6d2f65bd0148'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.119270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbd2c2b3b6f418da7e06ad768ea5d37', 'request_id': 'req-c48ab82a-1110-4f32-894a-6d2f65bd0148'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.129748 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] The token cache is being invalidate because project 5fbd2c2b3b6f418da7e06ad768ea5d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:21.130052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.130653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.132201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1246/2487] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:21 2022] DELETE /identity/v3/projects/5fbd2c2b3b6f418da7e06ad768ea5d37 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:21.136225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.146492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4167c3-24a8-4adc-9bc6-d408158cad98 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.166483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:21.205629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.206523 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.206844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.207125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] PATH_INFO: `/v3/users/4f9a5cc076d644589a02f7b489fe278e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.208407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9a5cc076d644589a02f7b489fe278e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.210404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:21.255854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9a5cc076d644589a02f7b489fe278e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.262828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9a5cc076d644589a02f7b489fe278e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.263434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9a5cc076d644589a02f7b489fe278e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.275402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9a5cc076d644589a02f7b489fe278e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.276016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3029ee88-dff0-49d0-b9e0-7cbcfe961277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9a5cc076d644589a02f7b489fe278e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.287574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1242/2488] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:21 2022] DELETE /identity/v3/users/4f9a5cc076d644589a02f7b489fe278e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:21.551870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.553301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:21.567357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ab82a-1110-4f32-894a-6d2f65bd0148 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.623828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.624686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.624967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.625308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] PATH_INFO: `/v3/projects/0b95465d3800437d816572fb656576ff` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.627051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b95465d3800437d816572fb656576ff)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.628996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:21.676917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b95465d3800437d816572fb656576ff', 'request_id': 'req-3653ada7-7dd4-4029-95a2-53d292018afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.682932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b95465d3800437d816572fb656576ff', 'request_id': 'req-3653ada7-7dd4-4029-95a2-53d292018afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.691107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b95465d3800437d816572fb656576ff', 'request_id': 'req-3653ada7-7dd4-4029-95a2-53d292018afc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.701362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] The token cache is being invalidate because project 0b95465d3800437d816572fb656576ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:21.701648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.702283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3653ada7-7dd4-4029-95a2-53d292018afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:21.703814 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1247/2489] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:21 2022] DELETE /identity/v3/projects/0b95465d3800437d816572fb656576ff => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:21.708582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.708871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.709148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.727051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:21.728200 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:21.786788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1243/2490] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:21.793981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.794257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.794535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.814700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:21.816055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:21.823912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1248/2491] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:21.828005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.837624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb68af-97cd-4844-b2cc-418f612a83bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.842197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.842876 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.843171 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.843453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.844183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:21.844766 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.850127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:21.862475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1244/2492] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:16:21.866743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.876426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76980b49-4e41-4152-8793-4bb0f6030237 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.881021 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.881665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.881946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.882225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.882934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.884823 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:21.909322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1249/2493] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:21 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:21.913389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:21.924044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b8805-dc77-40b7-a93a-35eeaf03865c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:21.953436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:21.982846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:21.983446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:21.983726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:21.984000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:21.985272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:21.987207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.006578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1245/2494] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:21 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:22.010972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.021209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e24dc6-f4e8-4c0e-8d2b-d07093a705b4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.025750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.026390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.026668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.026969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.027680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.029261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.039874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1250/2495] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:22.043737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.055242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33422c98-28bb-4f15-b777-4cac85740749 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.060072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.060724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.061009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.061288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.062012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.063640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c33755-f3b1-477f-9c25-7c8c5d5d0bfd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.073222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1246/2496] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:22.077235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.087362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06142813-9609-4028-b69b-43d45a1d0ca6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.091897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.092754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.093032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.093304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] PATH_INFO: `/v3/projects/6c0c616f93b844c8a21fc1d9e1dfa0f3/users/363fc9aaf5f44a2da5a6f9f767855572/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.096824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:22.110372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0c616f93b844c8a21fc1d9e1dfa0f3, user_id=363fc9aaf5f44a2da5a6f9f767855572, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.113071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.124973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1251/2497] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:22 2022] PUT /identity/v3/projects/6c0c616f93b844c8a21fc1d9e1dfa0f3/users/363fc9aaf5f44a2da5a6f9f767855572/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:22.135134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.135429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.135717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.161720 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] MFA Rules not processed for user `363fc9aaf5f44a2da5a6f9f767855572`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:22.162981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:22.225753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1247/2498] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:22.230338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.290629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3857a13-1eb9-4d4f-b6ee-c7f84d0b5cc8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.295124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.295770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.296051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.296329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.297551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.299489 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.323618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1252/2499] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:22 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:22.327536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.337856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad722-d67f-4194-ade6-cecbe9dc2649 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.393130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.393783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.394065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.394342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.395048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.396958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.416810 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1248/2500] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:22 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:22.420764 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.431042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1faf9-eb56-4066-a989-63c9c51cb3a9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.435840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.436485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.436765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.437037 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.439227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.439580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.450140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1253/2501] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:22.454666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.464807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a9493-7416-4168-83c5-e8a955533d50 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.470207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.471164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.471448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.471727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] PATH_INFO: `/v3/projects/c034886fd6b44c29ab8002a02fc46916/users/ba69192e02a74121b514eabe1f3aa153/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.486267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c034886fd6b44c29ab8002a02fc46916, user_id=ba69192e02a74121b514eabe1f3aa153, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.488962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.501363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1249/2502] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:22 2022] PUT /identity/v3/projects/c034886fd6b44c29ab8002a02fc46916/users/ba69192e02a74121b514eabe1f3aa153/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:22.505201 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.515273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc693b53-1f61-4021-8b53-66e71c72b75b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.569716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.570299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.570580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.570910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.571624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.573205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84786ad-87b6-43a0-a53e-104b280e15b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.583125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1254/2503] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:22.587293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.593913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:22.598419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca01a23-4a29-4a77-8f1a-ca7781026d01 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.603975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.604844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.605123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.605394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] PATH_INFO: `/v3/projects/c034886fd6b44c29ab8002a02fc46916/users/ba69192e02a74121b514eabe1f3aa153/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.607368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c034886fd6b44c29ab8002a02fc46916, user_id=ba69192e02a74121b514eabe1f3aa153, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.610056 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.620026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1250/2504] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:22 2022] PUT /identity/v3/projects/c034886fd6b44c29ab8002a02fc46916/users/ba69192e02a74121b514eabe1f3aa153/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:22.630756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.631081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.631360 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.657429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] MFA Rules not processed for user `ba69192e02a74121b514eabe1f3aa153`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:22.658612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:22.700853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:22.726029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1255/2505] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:22.737295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.739030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:22.795944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf7aa9-047b-4593-b2e3-04b90c969c55 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.799706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.800349 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.800635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.800913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.810353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.811832 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.821929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1251/2506] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:22.898810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:22.908535 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7235bae-9996-43db-a494-0b3e220d572d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:22.912237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:22.912869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:22.913150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:22.913424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:22.971020 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:22.972474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655748c-ebb5-44fe-8afa-330c138eea09 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:22.981747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1256/2507] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:22 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:25.675512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:25.737278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aef5f9-5968-4398-9097-6fafbaef10ce service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:25.741753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:25.742610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:25.742927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:25.743208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] PATH_INFO: `/v3/users/363fc9aaf5f44a2da5a6f9f767855572` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:25.744383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=363fc9aaf5f44a2da5a6f9f767855572)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:25.746283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:25.792943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363fc9aaf5f44a2da5a6f9f767855572'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:25.800570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363fc9aaf5f44a2da5a6f9f767855572'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:25.801192 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363fc9aaf5f44a2da5a6f9f767855572'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:25.815949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363fc9aaf5f44a2da5a6f9f767855572'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:25.816839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363fc9aaf5f44a2da5a6f9f767855572'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:25.838318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1252/2508] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:25 2022] DELETE /identity/v3/users/363fc9aaf5f44a2da5a6f9f767855572 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:25.842828 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:25.843307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:25.843546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:25.860723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:25.861928 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:25.918378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1257/2509] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:25.928055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:25.929576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:25.944706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:25.988411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4e9d8-a07b-4577-a589-6a062decf689 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:25.992072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:25.992706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:25.992988 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:25.993265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:26.002710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:26.004167 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:26.013602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1253/2510] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:26.289263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:26.290808 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:26.303674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45a188-ad66-4ed7-990f-e0d472062191 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:26.308563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:26.310685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:26.311105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:26.311381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] PATH_INFO: `/v3/projects/6c0c616f93b844c8a21fc1d9e1dfa0f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:26.313290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0c616f93b844c8a21fc1d9e1dfa0f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:26.316857 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:26.364030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0c616f93b844c8a21fc1d9e1dfa0f3', 'request_id': 'req-9a5a9389-85ac-4bca-b791-df02acc5149b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.370157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0c616f93b844c8a21fc1d9e1dfa0f3', 'request_id': 'req-9a5a9389-85ac-4bca-b791-df02acc5149b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.378181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0c616f93b844c8a21fc1d9e1dfa0f3', 'request_id': 'req-9a5a9389-85ac-4bca-b791-df02acc5149b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.388527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] The token cache is being invalidate because project 6c0c616f93b844c8a21fc1d9e1dfa0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:26.388810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.389418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.390950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1258/2511] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:26 2022] DELETE /identity/v3/projects/6c0c616f93b844c8a21fc1d9e1dfa0f3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:26.394980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:26.404489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1433b90-36fa-497e-995d-95e55f311184 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:26.464051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:26.464905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:26.465185 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:26.465455 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] PATH_INFO: `/v3/users/ba69192e02a74121b514eabe1f3aa153` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:26.466618 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba69192e02a74121b514eabe1f3aa153)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:26.468587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:26.514022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba69192e02a74121b514eabe1f3aa153'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.522187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba69192e02a74121b514eabe1f3aa153'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.522828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba69192e02a74121b514eabe1f3aa153'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.534631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba69192e02a74121b514eabe1f3aa153'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.535281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5fb3c8fe-d939-4a8d-a7a6-ddfea786d2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba69192e02a74121b514eabe1f3aa153'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.547540 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1254/2512] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:26 2022] DELETE /identity/v3/users/ba69192e02a74121b514eabe1f3aa153 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:26.817951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:26.819800 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:26.829414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:26.836352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a9389-85ac-4bca-b791-df02acc5149b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:26.899545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:26.900399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:26.900732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:26.901009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] PATH_INFO: `/v3/projects/c034886fd6b44c29ab8002a02fc46916` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:26.902417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c034886fd6b44c29ab8002a02fc46916)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:26.904329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:26.950404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c034886fd6b44c29ab8002a02fc46916', 'request_id': 'req-e5ca68ba-895b-4215-aa31-d0ecb10ae529'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.957263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c034886fd6b44c29ab8002a02fc46916', 'request_id': 'req-e5ca68ba-895b-4215-aa31-d0ecb10ae529'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.964763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c034886fd6b44c29ab8002a02fc46916', 'request_id': 'req-e5ca68ba-895b-4215-aa31-d0ecb10ae529'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.975885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] The token cache is being invalidate because project c034886fd6b44c29ab8002a02fc46916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:26.976175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.976794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e5ca68ba-895b-4215-aa31-d0ecb10ae529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:26.978342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1259/2513] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:26 2022] DELETE /identity/v3/projects/c034886fd6b44c29ab8002a02fc46916 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:26.983217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:26.983714 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:26.983817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.001821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:27.003122 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:27.035323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:27.068504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1255/2514] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:16:27.076273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.076561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.076839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.099577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:27.100810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:27.108903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1260/2515] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:27.112948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.122597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61116c-399d-4189-b50e-cf62f28f424f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.127574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.128231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.128513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.128787 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.129520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:27.129849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.131612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.145474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1256/2516] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 01 07:16:27.149867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.160035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96a144-8841-4b4b-8bb8-9bdb3b095e0a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.164630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.165283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.165563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.165835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.166510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.168438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.195025 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1261/2517] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:27.198986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.212319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69899d98-8e3f-4fb2-a872-2600d1a4915d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.275405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.276083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.276369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.276643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.277348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.279293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.301402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1257/2518] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:27.302615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.313690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8aca-c5e7-464f-ac29-1b409f1122d4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.320636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.321756 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.322256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.322746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.324160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.327334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.347319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1262/2519] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:16:27.351114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.361333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e5b5-7260-46cd-b856-edd8a3945071 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.366010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.366677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.366985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.367266 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.368024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.369621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53838ba-60af-4231-842a-96dc9a92ede9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.380142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1258/2520] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:27.384195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.394854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bd541-e989-4f4f-a0e3-d87a39631bae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.399639 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.400632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.401034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.401382 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] PATH_INFO: `/v3/projects/c95d31605b48497b912ad2f3d02b8d7a/users/a8532f650f744858a06243122456bce8/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.415989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95d31605b48497b912ad2f3d02b8d7a, user_id=a8532f650f744858a06243122456bce8, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.418795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.431484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1263/2521] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:27 2022] PUT /identity/v3/projects/c95d31605b48497b912ad2f3d02b8d7a/users/a8532f650f744858a06243122456bce8/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:27.442687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.443003 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.443289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.469754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] MFA Rules not processed for user `a8532f650f744858a06243122456bce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:27.471000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:27.535143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1259/2522] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:27.539534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.571993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:27.608162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffeea3-1a73-45c5-9dab-29952bee4273 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.613350 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.614001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.614283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.614560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.615272 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.617177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.641440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1264/2523] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:27.645662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.656345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9365ed-a31a-4cdf-a30f-43a70da2dd44 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.684996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:27.712957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.713623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.713908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.714184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.714909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.716839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.738157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1260/2524] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:27.741275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.752110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad41270-2ba0-4aef-bd27-08574bc08c24 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.756711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.757364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.757647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.757922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.758652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.760285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.770884 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1265/2525] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:27.775158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.785005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3cc1d0-d0fa-4a1b-b744-b98ee93640ab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.789610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.790486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.790785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.791081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] PATH_INFO: `/v3/projects/a49bb84ee56b481b8cea1df090b78c66/users/931d34b88fc74ab5bf4ce2695b0d2f88/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.807615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49bb84ee56b481b8cea1df090b78c66, user_id=931d34b88fc74ab5bf4ce2695b0d2f88, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.810661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.824926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1261/2526] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:27 2022] PUT /identity/v3/projects/a49bb84ee56b481b8cea1df090b78c66/users/931d34b88fc74ab5bf4ce2695b0d2f88/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:27.829023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.839587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5bd32-9396-4904-894c-2a4b619aa868 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.895648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.896318 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.896603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.896883 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.897604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.899253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a7383-c472-44b9-a3be-d22a6df8100a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.908936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1266/2527] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:27.913065 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:27.923874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53cdb4-8297-41f0-a88a-035f8aa6b4dc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:27.928512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:27.929379 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.929657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.929927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] PATH_INFO: `/v3/projects/a49bb84ee56b481b8cea1df090b78c66/users/931d34b88fc74ab5bf4ce2695b0d2f88/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.931941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49bb84ee56b481b8cea1df090b78c66, user_id=931d34b88fc74ab5bf4ce2695b0d2f88, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:27.934867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:27.946450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1262/2528] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:27 2022] PUT /identity/v3/projects/a49bb84ee56b481b8cea1df090b78c66/users/931d34b88fc74ab5bf4ce2695b0d2f88/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:27.957938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:27.958236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:27.958518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:27.985758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] MFA Rules not processed for user `931d34b88fc74ab5bf4ce2695b0d2f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:27.987000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:28.052729 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1267/2529] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:28.063800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:28.066035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:28.122845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18784-2001-4c16-8b43-e5700562a2f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:28.126486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:28.127151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:28.127438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:28.127717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:28.136806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:28.138227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbfcbc-e441-4f09-a082-9ff156489a3a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:28.147680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1263/2530] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:28.703949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:28.730357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:28.768646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d8343-e643-4ffb-abcd-55d59c60e0fa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:28.776522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:28.777393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:28.777672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:28.777945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] PATH_INFO: `/v3/users/a8532f650f744858a06243122456bce8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:28.779268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8532f650f744858a06243122456bce8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:28.781219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:28.826911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8532f650f744858a06243122456bce8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:28.838843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8532f650f744858a06243122456bce8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:28.839462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8532f650f744858a06243122456bce8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:28.853133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8532f650f744858a06243122456bce8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:28.853792 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8532f650f744858a06243122456bce8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:28.865694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1268/2531] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:28 2022] DELETE /identity/v3/users/a8532f650f744858a06243122456bce8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:28.870303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:28.870596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:28.870906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:28.888698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:28.889908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:28.946835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1264/2532] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:28.956917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:28.958322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:29.014891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e59003-4379-4adf-8228-bfd5718d529d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:29.019047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:29.019686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:29.019969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:29.020244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:29.029075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:29.030496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:29.039744 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1269/2533] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:29.302530 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:29.304051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:29.316901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc60d3-e830-41c1-b824-012babac2e76 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:29.322297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:29.323204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:29.323513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:29.323793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] PATH_INFO: `/v3/projects/c95d31605b48497b912ad2f3d02b8d7a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:29.324975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95d31605b48497b912ad2f3d02b8d7a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:29.326930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:29.350049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:29.373750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95d31605b48497b912ad2f3d02b8d7a', 'request_id': 'req-2fbe2e02-3e11-4993-a631-bed4934dbaee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.381502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95d31605b48497b912ad2f3d02b8d7a', 'request_id': 'req-2fbe2e02-3e11-4993-a631-bed4934dbaee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.387731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95d31605b48497b912ad2f3d02b8d7a', 'request_id': 'req-2fbe2e02-3e11-4993-a631-bed4934dbaee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.397860 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] The token cache is being invalidate because project c95d31605b48497b912ad2f3d02b8d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:29.398144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.398756 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.400506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1265/2534] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:29 2022] DELETE /identity/v3/projects/c95d31605b48497b912ad2f3d02b8d7a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:29.404747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:29.414451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8647b-9305-41d4-8a3f-3dfdb31d650c service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:29.457537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:29.472399 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:29.473270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:29.473547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:29.473818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] PATH_INFO: `/v3/users/931d34b88fc74ab5bf4ce2695b0d2f88` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:29.475015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=931d34b88fc74ab5bf4ce2695b0d2f88)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:29.476997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:29.522433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '931d34b88fc74ab5bf4ce2695b0d2f88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.529422 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '931d34b88fc74ab5bf4ce2695b0d2f88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.530048 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '931d34b88fc74ab5bf4ce2695b0d2f88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.542291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '931d34b88fc74ab5bf4ce2695b0d2f88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.542969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8696daef-d885-440c-a1c9-2e1ca4901533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '931d34b88fc74ab5bf4ce2695b0d2f88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.555715 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1270/2535] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:29 2022] DELETE /identity/v3/users/931d34b88fc74ab5bf4ce2695b0d2f88 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:29.824443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:29.825987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:29.841058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe2e02-3e11-4993-a631-bed4934dbaee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:29.898877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:29.899730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:29.900010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:29.900285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] PATH_INFO: `/v3/projects/a49bb84ee56b481b8cea1df090b78c66` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:29.901460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49bb84ee56b481b8cea1df090b78c66)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:29.903404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:29.947606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49bb84ee56b481b8cea1df090b78c66', 'request_id': 'req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.954760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49bb84ee56b481b8cea1df090b78c66', 'request_id': 'req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.960744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49bb84ee56b481b8cea1df090b78c66', 'request_id': 'req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.971687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] The token cache is being invalidate because project a49bb84ee56b481b8cea1df090b78c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:29.971974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.972601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb4230cf-0cb7-4587-8f11-1b8424eeaa13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:29.974130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1266/2536] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:29 2022] DELETE /identity/v3/projects/a49bb84ee56b481b8cea1df090b78c66 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:29.979125 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:29.979604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:29.979722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:29.997081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:29.998315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:30.057180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1271/2537] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:30.064869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.065156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.065436 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.083614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:30.084792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:30.092476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1267/2538] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:30.096942 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.106446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645da0b-bf7e-4aad-ba0e-1dc19c36557f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.111127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.111778 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.112066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.112347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.113073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:30.113384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.115001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.126603 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1272/2539] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:16:30.130758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.139696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb0b30-447a-4b11-9167-fddb0a151395 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.144847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.145475 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.145755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.146027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.146706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.148625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.174224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1268/2540] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:30.178209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.188321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64f148-aa75-4f2a-a1a8-948d243c704d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.237259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:30.247559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.248228 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.248560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.248811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.249500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.251971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.271994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1273/2541] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:30.275980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.286206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbcf44-6aa0-4751-997a-a7f206ce0d19 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.290838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.291485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.291771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.292048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.292771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.294359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.299472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:30.305599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1269/2542] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:30.309265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.319804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dfede-55cc-4472-9c88-16069195b2a6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.324403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.325055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.325358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.325628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.326353 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.327961 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda414e-4b21-4569-81a5-a1669bc323f4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.338234 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1274/2543] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:30.342142 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.352339 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0cf51-0020-478f-bd47-12e4b83fd86b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.357110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.357980 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.358263 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.358538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] PATH_INFO: `/v3/projects/faa98b8e088443f49c033fa51f7a1239/users/f2949218a692493e9550ffe986c51abb/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.373023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa98b8e088443f49c033fa51f7a1239, user_id=f2949218a692493e9550ffe986c51abb, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.376257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.388702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1270/2544] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:30 2022] PUT /identity/v3/projects/faa98b8e088443f49c033fa51f7a1239/users/f2949218a692493e9550ffe986c51abb/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:30.399379 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.399714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.399996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.425256 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] MFA Rules not processed for user `f2949218a692493e9550ffe986c51abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:30.426453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:30.489622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1275/2545] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:30.494275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.554402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462992ce-74c0-469f-8438-a2be8f4ce3a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.559197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.559847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.560128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.560404 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.561078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.563538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.588226 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1271/2546] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:30.592235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.602863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff0d75-9606-494f-b792-4c620cc2439b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.658597 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.659284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.659569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.659844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.660530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.662449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.681812 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1276/2547] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:30.686041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.696853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8872f7-4e32-407f-b999-32363ba734f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.701461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.702105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.702389 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.702664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.703438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.705021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.715382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1272/2548] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:30.719269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.729136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe85ed-1388-4fa4-8bb9-ad7db4a0ffab admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.733659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.735092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.735370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.735648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] PATH_INFO: `/v3/projects/1f90c4cd83714da69610cb420a0b22e0/users/dabb7154e09b4e87bb12c1f67e7dca4a/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.750862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f90c4cd83714da69610cb420a0b22e0, user_id=dabb7154e09b4e87bb12c1f67e7dca4a, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.753483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.765786 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1277/2549] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:30 2022] PUT /identity/v3/projects/1f90c4cd83714da69610cb420a0b22e0/users/dabb7154e09b4e87bb12c1f67e7dca4a/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:30.771208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.781965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74856a89-37c6-4f6f-b8e4-0f67af0fa44d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.838535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.839138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.839424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.839700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.840426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.842074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dc014-42ff-4285-9cb1-f46d71a8aaef admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.851681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1273/2550] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:30.855804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:30.866413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc77f63-d52b-4b85-a057-e224a7f31d42 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:30.870968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:30.871835 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.872113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.872385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] PATH_INFO: `/v3/projects/1f90c4cd83714da69610cb420a0b22e0/users/dabb7154e09b4e87bb12c1f67e7dca4a/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.874319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f90c4cd83714da69610cb420a0b22e0, user_id=dabb7154e09b4e87bb12c1f67e7dca4a, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:30.877575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:30.888334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1278/2551] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:30 2022] PUT /identity/v3/projects/1f90c4cd83714da69610cb420a0b22e0/users/dabb7154e09b4e87bb12c1f67e7dca4a/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:16:30.952754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:30.953030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:30.953347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:30.979532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] MFA Rules not processed for user `dabb7154e09b4e87bb12c1f67e7dca4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:30.980761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:31.014616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:31.050597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1274/2552] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:31.061970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:31.063711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:31.078547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:31.122533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d2cc8-3707-4b11-8088-3062ed0ce775 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:31.126251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:31.126918 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:31.127205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:31.127482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:31.136400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:31.137831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:31.147198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1279/2553] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:31.220382 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:31.230206 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259223dc-3b72-436f-bdfa-8aa242873c04 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:31.233862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:31.235186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:31.235471 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:31.235745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:31.291972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:31.293390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8ba5-ba4c-44a2-b22f-fbd0cdaddc68 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:31.302657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1275/2554] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:31.557661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:31.623479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e581b-0b55-4ab1-ad72-75ad2960cd63 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:31.628087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:31.628894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:31.629180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:31.629457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] PATH_INFO: `/v3/users/f2949218a692493e9550ffe986c51abb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:31.631198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2949218a692493e9550ffe986c51abb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:31.633135 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:31.680043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2949218a692493e9550ffe986c51abb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:31.687825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2949218a692493e9550ffe986c51abb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:31.689010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2949218a692493e9550ffe986c51abb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:31.700919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2949218a692493e9550ffe986c51abb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:31.701558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2949218a692493e9550ffe986c51abb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:31.714112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1280/2555] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:31 2022] DELETE /identity/v3/users/f2949218a692493e9550ffe986c51abb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:31.718574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:31.719100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:31.719208 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:31.736544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:31.737748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:31.797609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1276/2556] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:16:31.808267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:31.810649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:31.852358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:31.871197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a736c-d5f5-4c17-a79a-b2e48db52e0d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:31.874790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:31.875441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:31.876224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:31.876495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:31.885782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:31.887187 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:31.897012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1281/2557] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:32.160993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:32.162421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:32.174819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f675086-15da-4e4d-b53f-9b52a4451e9d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:32.179369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:32.180217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.180500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.180773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] PATH_INFO: `/v3/projects/faa98b8e088443f49c033fa51f7a1239` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.181955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa98b8e088443f49c033fa51f7a1239)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:32.183912 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:32.231270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa98b8e088443f49c033fa51f7a1239', 'request_id': 'req-5b69117c-8f3c-41e0-b771-6edd6964a543'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.238590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa98b8e088443f49c033fa51f7a1239', 'request_id': 'req-5b69117c-8f3c-41e0-b771-6edd6964a543'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.244920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa98b8e088443f49c033fa51f7a1239', 'request_id': 'req-5b69117c-8f3c-41e0-b771-6edd6964a543'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.248669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:32.258688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] The token cache is being invalidate because project faa98b8e088443f49c033fa51f7a1239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:32.259136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.259902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.261421 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1277/2558] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:32 2022] DELETE /identity/v3/projects/faa98b8e088443f49c033fa51f7a1239 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:32.265459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:32.274887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a8e0-2cd6-4568-b999-12895da71398 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:32.335165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:32.336033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.336320 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.336604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] PATH_INFO: `/v3/users/dabb7154e09b4e87bb12c1f67e7dca4a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.337783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] RBAC: Authorizing `identity:delete_user(user_id=dabb7154e09b4e87bb12c1f67e7dca4a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:32.339745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:32.384229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dabb7154e09b4e87bb12c1f67e7dca4a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.391377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dabb7154e09b4e87bb12c1f67e7dca4a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.391993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dabb7154e09b4e87bb12c1f67e7dca4a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.403717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dabb7154e09b4e87bb12c1f67e7dca4a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.404290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-84878931-332d-4cff-8fbd-5caa049dc3de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dabb7154e09b4e87bb12c1f67e7dca4a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.416182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1282/2559] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:16:32 2022] DELETE /identity/v3/users/dabb7154e09b4e87bb12c1f67e7dca4a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:32.685873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:32.687331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:32.700141 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69117c-8f3c-41e0-b771-6edd6964a543 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:32.757603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:32.758456 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.758738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.759041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] PATH_INFO: `/v3/projects/1f90c4cd83714da69610cb420a0b22e0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.760224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f90c4cd83714da69610cb420a0b22e0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:32.762094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:32.808267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f90c4cd83714da69610cb420a0b22e0', 'request_id': 'req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.815981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f90c4cd83714da69610cb420a0b22e0', 'request_id': 'req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.822096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f90c4cd83714da69610cb420a0b22e0', 'request_id': 'req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.832875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] The token cache is being invalidate because project 1f90c4cd83714da69610cb420a0b22e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:16:32.833161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.833786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4b85e5ad-74f3-4cf9-811e-1c9baa0060b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:16:32.835366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1278/2560] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:16:32 2022] DELETE /identity/v3/projects/1f90c4cd83714da69610cb420a0b22e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:32.841929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.842245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.842530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.860815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:32.862015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:32.914907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:32.926887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1283/2561] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:32.933074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.933309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.933592 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.951984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:32.953175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:32.961444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1279/2562] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:16:32.965505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:32.975793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f16017-9188-41cf-a194-82293ac42ff3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:32.980359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:32.981006 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:32.981290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:32.981566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:32.982282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:16:32.982588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:32.984219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:32.995697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1284/2563] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:16:32.999606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.008679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e947619-0b6b-4229-94ab-59d66c0989bd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.013280 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.013911 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.014190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.014462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.015182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.017591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.043673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1280/2564] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:16:32 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:16:33.048420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.060250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414366a-4d95-4e48-9302-bb5d30774c8a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.115207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.115863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.116147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.116421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.117104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.119047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.137954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1285/2565] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:16:33 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:16:33.142494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.152950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92e751-9ead-4350-9997-978e0a420a9e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.158762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.159438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.159725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.160030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.160758 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.162346 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.172617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1281/2566] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:33.176726 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.186818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf429d43-aa86-4c1b-ab88-43c15a33c310 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.192109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.192795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.193079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.193357 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.194113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.195901 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd583d-1365-4c6f-a53c-027c398773fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.206537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1286/2567] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:16:33.210536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.220557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38aa18-2f8b-4a6c-88cc-87bd08f4b5d7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.225277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.226177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.226460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.226817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] PATH_INFO: `/v3/projects/e8493245b5e64463b1bc8551dda0e04e/users/e0b3996b750d4cf8b04843baa4dc3a88/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.240965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8493245b5e64463b1bc8551dda0e04e, user_id=e0b3996b750d4cf8b04843baa4dc3a88, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.243651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.256719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1282/2568] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:16:33 2022] PUT /identity/v3/projects/e8493245b5e64463b1bc8551dda0e04e/users/e0b3996b750d4cf8b04843baa4dc3a88/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:16:33.268537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.269005 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.269101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.294844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] MFA Rules not processed for user `e0b3996b750d4cf8b04843baa4dc3a88`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:16:33.296064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:33.358356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1287/2569] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:16:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:16:33.369575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:16:33.371265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:16:33.393833 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:33.428667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060df367-8ea4-4c98-ab2c-5fdf7d1b2ed0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:16:33.432279 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d06e6ad60240329d83bc4767439ee5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:16:33.432913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:16:33.433197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:16:33.433476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:16:33.442542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:16:33.443947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5be2c-5e7f-4f4f-b963-430a7e3c163b service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:16:33.452704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1283/2570] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:16:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:16:43.355476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:16:43.448692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=17 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:04.021046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:04.022496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:04.104885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9174fa-e7e7-4758-819f-ffced8090dc7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:04.112621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:04.112621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:04.112621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:04.112621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:04.114385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:04.127053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:04.129245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:04.140274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1288/2571] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:17:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:17:06.828385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72368a-2739-46a9-92df-e88a919502fc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:06.828893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72368a-2739-46a9-92df-e88a919502fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:06.829000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72368a-2739-46a9-92df-e88a919502fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:06.846128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6e72368a-2739-46a9-92df-e88a919502fc None None] MFA Rules not processed for user `df44f27783d34370ad83470ffde558a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:17:06.847996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e72368a-2739-46a9-92df-e88a919502fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:06.903727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1284/2572] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:17:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:17:14.135860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:16.896884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:20.287680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:20.362163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2290a0-60d2-42c9-9841-329b17104f7f service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:20.368075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:20.368976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:20.369254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:20.369534 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] PATH_INFO: `/v3/users/e0b3996b750d4cf8b04843baa4dc3a88` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:20.370802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b3996b750d4cf8b04843baa4dc3a88)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:20.372802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:20.430009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b3996b750d4cf8b04843baa4dc3a88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.438504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b3996b750d4cf8b04843baa4dc3a88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.439237 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b3996b750d4cf8b04843baa4dc3a88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.452560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b3996b750d4cf8b04843baa4dc3a88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.453270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b3996b750d4cf8b04843baa4dc3a88'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.466261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1289/2573] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:17:20 2022] DELETE /identity/v3/users/e0b3996b750d4cf8b04843baa4dc3a88 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:20.470797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:20.471277 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:20.471478 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:20.490072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:17:20.491827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:20.553975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1285/2574] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:17:20.563683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:20.565079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:20.621032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032349da-2a51-4326-80b9-9df217d0a47a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:20.624759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:20.625392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:20.625674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:20.625946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:20.635233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:20.636656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:20.646504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1290/2575] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:17:20.907967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:20.909441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:20.921369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e498826-aa38-421d-a685-345f7f990bc9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:20.926899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:20.928308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:20.928806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:20.929204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] PATH_INFO: `/v3/projects/e8493245b5e64463b1bc8551dda0e04e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:20.931080 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8493245b5e64463b1bc8551dda0e04e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:20.934410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:20.982054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8493245b5e64463b1bc8551dda0e04e', 'request_id': 'req-5e722861-e46e-40fd-ab5e-730b04e7c817'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.989997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8493245b5e64463b1bc8551dda0e04e', 'request_id': 'req-5e722861-e46e-40fd-ab5e-730b04e7c817'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:20.996688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8493245b5e64463b1bc8551dda0e04e', 'request_id': 'req-5e722861-e46e-40fd-ab5e-730b04e7c817'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:21.007522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] The token cache is being invalidate because project e8493245b5e64463b1bc8551dda0e04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:17:21.007809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:21.008448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:21.009985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1286/2576] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:17:20 2022] DELETE /identity/v3/projects/e8493245b5e64463b1bc8551dda0e04e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:17:30.641751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:31.003942 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:33.126359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:33.127797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:33.139486 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5b59c-2c0a-4705-81a3-b826bd22325d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:33.192033 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:33.193081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:33.193531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:33.193991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:33.196183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:33.259829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:33.261173 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:33.270580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1291/2577] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:17:39.699708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:39.701158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:39.717485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722861-e46e-40fd-ab5e-730b04e7c817 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:39.723022 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:39.723897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:39.724748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:39.725030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] PATH_INFO: `/v3/users/0a36e02c916b494180e68d3f231ee714` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:39.726364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a36e02c916b494180e68d3f231ee714)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:39.729058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:39.777977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a36e02c916b494180e68d3f231ee714'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:39.785270 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a36e02c916b494180e68d3f231ee714'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:39.785874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a36e02c916b494180e68d3f231ee714'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:39.799327 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a36e02c916b494180e68d3f231ee714'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:39.800017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a36e02c916b494180e68d3f231ee714'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:39.812402 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1287/2578] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:17:39 2022] DELETE /identity/v3/users/0a36e02c916b494180e68d3f231ee714 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:39.980060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:39.992209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd23c-d32b-4aa4-a88e-d06e27daf1c5 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:40.049749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:40.050695 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:40.051229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:40.051547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] PATH_INFO: `/v3/projects/d8def43018744d7b8d5a5f422e44bf89` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:40.053055 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8def43018744d7b8d5a5f422e44bf89)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:40.055197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:40.100751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8def43018744d7b8d5a5f422e44bf89', 'request_id': 'req-e6344cc7-d23a-477c-85c1-24dde3c2e69f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.102966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:40.107242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8def43018744d7b8d5a5f422e44bf89', 'request_id': 'req-e6344cc7-d23a-477c-85c1-24dde3c2e69f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.114630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8def43018744d7b8d5a5f422e44bf89', 'request_id': 'req-e6344cc7-d23a-477c-85c1-24dde3c2e69f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.124783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] The token cache is being invalidate because project d8def43018744d7b8d5a5f422e44bf89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:17:40.125076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.125699 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.127289 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1292/2579] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:17:39 2022] DELETE /identity/v3/projects/d8def43018744d7b8d5a5f422e44bf89 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:40.131709 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:40.133098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:40.146579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e542f94-e666-4ada-9a81-ca76ff66c3ae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:40.203921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:40.204780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:40.205060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:40.205334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] PATH_INFO: `/v3/users/af8f9521b12d464cbd49422a77842b20` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:40.206495 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8f9521b12d464cbd49422a77842b20)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:40.208730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:40.255203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8f9521b12d464cbd49422a77842b20'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.262839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8f9521b12d464cbd49422a77842b20'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.263448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8f9521b12d464cbd49422a77842b20'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.275019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8f9521b12d464cbd49422a77842b20'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.275643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8f9521b12d464cbd49422a77842b20'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.277555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:40.290107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1288/2580] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:17:40 2022] DELETE /identity/v3/users/af8f9521b12d464cbd49422a77842b20 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:40.466542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:40.468673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:40.487008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6344cc7-d23a-477c-85c1-24dde3c2e69f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:40.547479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:40.548352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:40.548636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:40.548910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] PATH_INFO: `/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:40.550117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d82e6fabd448d99fa6e8d56c17cf0e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:40.552084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:40.598457 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d82e6fabd448d99fa6e8d56c17cf0e', 'request_id': 'req-3919937a-2329-4573-b937-c8a84d9df6be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.605076 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d82e6fabd448d99fa6e8d56c17cf0e', 'request_id': 'req-3919937a-2329-4573-b937-c8a84d9df6be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.611935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d82e6fabd448d99fa6e8d56c17cf0e', 'request_id': 'req-3919937a-2329-4573-b937-c8a84d9df6be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.623365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] The token cache is being invalidate because project 03d82e6fabd448d99fa6e8d56c17cf0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:17:40.623650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.624279 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3919937a-2329-4573-b937-c8a84d9df6be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:40.626044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1293/2581] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:17:40 2022] DELETE /identity/v3/projects/03d82e6fabd448d99fa6e8d56c17cf0e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:41.946928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:41.949009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:42.028591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab8d7f-38fa-4204-9826-b1443efb2f74 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:42.101704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:42.102401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:42.102674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:42.102960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:42.104916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:42.119954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:42.122642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:42.135337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1289/2582] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:17:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:17:48.199846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:48.200333 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:48.200473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:48.219118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:17:48.220335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:48.274221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1294/2583] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:17:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:17:48.321624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:48.323051 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:48.389741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b559-a2f6-4ae7-8a68-9e1ed420d27f service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:48.394629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:48.395290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:48.395573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:48.396375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:48.404332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:48.405892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:48.414527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1290/2584] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:17:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:17:48.774555 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:48.776049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:48.794451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:48.834471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d4d00-7e94-4959-acb8-f8b720c2aca0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:48.838053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:48.838693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:48.839019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:48.839299 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:48.841073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:48.902441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:48.903891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:48.913325 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1295/2585] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:17:55.053467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:55.055190 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:55.065130 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:55.071008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadda60b-8133-40ef-a601-717df3cd8993 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:55.075532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:55.076416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:55.076695 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:55.076971 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] PATH_INFO: `/v3/users/631f377cb70b454cbde19de83676f502` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:55.078144 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631f377cb70b454cbde19de83676f502)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:55.080114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:55.125643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631f377cb70b454cbde19de83676f502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.132450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631f377cb70b454cbde19de83676f502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.133232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631f377cb70b454cbde19de83676f502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.145005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631f377cb70b454cbde19de83676f502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.146153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631f377cb70b454cbde19de83676f502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.158162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1291/2586] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:17:55 2022] DELETE /identity/v3/users/631f377cb70b454cbde19de83676f502 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:55.328159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:55.339507 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fdcee2-ec8c-484b-b8c9-bd53ea3df04f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:55.394663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:55.396082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:55.396363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:55.396633 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] PATH_INFO: `/v3/projects/ce460f857d824071927f15e19cbb11e7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:55.397794 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce460f857d824071927f15e19cbb11e7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:55.399752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:55.444503 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce460f857d824071927f15e19cbb11e7', 'request_id': 'req-51262e67-2d53-4430-bb2f-c152e2406f6a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.450615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce460f857d824071927f15e19cbb11e7', 'request_id': 'req-51262e67-2d53-4430-bb2f-c152e2406f6a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.459425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce460f857d824071927f15e19cbb11e7', 'request_id': 'req-51262e67-2d53-4430-bb2f-c152e2406f6a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.464759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:55.472389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] The token cache is being invalidate because project ce460f857d824071927f15e19cbb11e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:17:55.472913 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.475047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.476816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1296/2587] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:17:55 2022] DELETE /identity/v3/projects/ce460f857d824071927f15e19cbb11e7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:55.480985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:55.482364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:55.495449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47516049-f05a-4742-a765-41f00d68a1f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:55.562561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:55.564479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:55.564479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:55.566868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] PATH_INFO: `/v3/users/741a67a1b8b3407bbc56aa695bb9c349` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:55.568997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=741a67a1b8b3407bbc56aa695bb9c349)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:55.572083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:55.628069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741a67a1b8b3407bbc56aa695bb9c349'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.639347 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741a67a1b8b3407bbc56aa695bb9c349'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.640009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741a67a1b8b3407bbc56aa695bb9c349'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.653262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741a67a1b8b3407bbc56aa695bb9c349'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.654006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d70bfa8f-650d-4a92-807b-b559ba3fad27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741a67a1b8b3407bbc56aa695bb9c349'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.659229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:17:55.667460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1292/2588] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:17:55 2022] DELETE /identity/v3/users/741a67a1b8b3407bbc56aa695bb9c349 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:17:55.847819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:17:55.849305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:17:55.862090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51262e67-2d53-4430-bb2f-c152e2406f6a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:17:55.922221 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:17:55.923118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:17:55.923401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:17:55.923674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] PATH_INFO: `/v3/projects/3faf9aa722c24908ab91db2d7449e758` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:17:55.924973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=3faf9aa722c24908ab91db2d7449e758)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:17:55.927527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:17:55.981448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3faf9aa722c24908ab91db2d7449e758', 'request_id': 'req-863b166c-11d6-4744-b5dc-0dfb50b4d8df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.989742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3faf9aa722c24908ab91db2d7449e758', 'request_id': 'req-863b166c-11d6-4744-b5dc-0dfb50b4d8df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:55.996022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3faf9aa722c24908ab91db2d7449e758', 'request_id': 'req-863b166c-11d6-4744-b5dc-0dfb50b4d8df'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:56.007057 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] The token cache is being invalidate because project 3faf9aa722c24908ab91db2d7449e758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:17:56.007345 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:56.007977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:17:56.010491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1297/2589] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:17:55 2022] DELETE /identity/v3/projects/3faf9aa722c24908ab91db2d7449e758 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:18:05.668501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:06.003344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:09.505620 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:09.506107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:09.506214 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:09.523969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:18:09.525170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:09.581877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1293/2590] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:18:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:18:09.593615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:09.595064 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:09.666169 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b166c-11d6-4744-b5dc-0dfb50b4d8df admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:09.671089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:09.671728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:09.672012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:09.672314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:09.680820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:09.682397 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:09.690500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1298/2591] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:18:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:18:15.482804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:15.484477 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:15.554759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9954001-824f-49ab-aa47-0e6db9c8aec0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:15.558716 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:15.559688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:15.559975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:15.560258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:15.562111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:15.642733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:15.644278 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a181dcf-34b6-4ca2-9817-fa7f0d1abfab service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:15.658376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1294/2592] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:18:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:18:19.218167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:19.228178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0f4e4-fe11-4318-a0b4-6cca68938f40 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:19.231713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:19.232337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:19.232620 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:19.232894 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:19.234641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:19.248078 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:19.249512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:19.260319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1299/2593] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:18:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:18:25.466893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:25.467380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:25.467480 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:25.484486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:18:25.485773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:25.493395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1295/2594] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:18:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:18:25.505665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:25.516771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb127aa-e819-45b7-b8a9-7fcd08d9f84a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:25.521847 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:25.522479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:25.522759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:25.523056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:25.531103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:25.532678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:25.541107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1300/2595] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:18:32.019282 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:32.020826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:32.037982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903437a-ea0f-45db-a8f5-aca083d75c43 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:32.043448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:32.044087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:32.044371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:32.044649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:32.046504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:32.093025 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:32.137098 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:32.138692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa2798-2366-4651-a4f2-d31d0c1df2ce service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:32.158094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1296/2596] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:18:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:18:35.537849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=44 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:42.155034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=23 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:44.472636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:44.482012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd26042-21e3-453d-a75e-b4854ddbec95 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:44.485625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:44.486260 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:44.486547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:44.486895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:44.488649 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:44.498091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:44.499562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:44.508044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1301/2597] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:18:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:18:46.020103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:46.020569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:46.020672 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:46.037950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:18:46.039169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:46.046283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1297/2598] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:18:46.059115 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:46.071689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb358c08-a5bd-47bc-98ca-a4dbe49d2932 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:46.076732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:46.077371 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:46.077652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:46.077927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:46.086182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:46.087799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:46.095711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1302/2599] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:18:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 01 07:18:56.033212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:56.092484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=12 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:18:58.560950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:18:58.562369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:58.573993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410873-d98b-4bbd-94b6-a893651d3086 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:18:58.577616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:18:58.578242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:18:58.578522 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:18:58.578854 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:18:58.580521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:18:58.646182 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:18:58.648368 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:18:58.657861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1298/2600] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:01.671090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:01.672634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:01.687853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3e4fd-320e-4cd8-b9a7-31a915ff2f2d service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:01.693717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:01.695428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:01.695807 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:01.696097 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] PATH_INFO: `/v3/users/df44f27783d34370ad83470ffde558a8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:01.712732 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df44f27783d34370ad83470ffde558a8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:01.714690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:01.757816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df44f27783d34370ad83470ffde558a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:01.764843 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df44f27783d34370ad83470ffde558a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:01.765468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df44f27783d34370ad83470ffde558a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:01.778028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df44f27783d34370ad83470ffde558a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:01.778662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-13bfe1c2-fa88-4593-993e-332ab5db7120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df44f27783d34370ad83470ffde558a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:01.790988 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1303/2601] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:01 2022] DELETE /identity/v3/users/df44f27783d34370ad83470ffde558a8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:01.965161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:01.975465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bdd5f-f2a5-4fa8-9528-7192258c4e39 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.032281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.033138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.033425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.033703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] PATH_INFO: `/v3/projects/dc0aa7206a704d45a610dccc24ec8767` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.043947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0aa7206a704d45a610dccc24ec8767)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.045848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.093006 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0aa7206a704d45a610dccc24ec8767', 'request_id': 'req-717ab50a-e86d-4849-9978-fa71a968f90e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:02.100035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0aa7206a704d45a610dccc24ec8767', 'request_id': 'req-717ab50a-e86d-4849-9978-fa71a968f90e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:02.106591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0aa7206a704d45a610dccc24ec8767', 'request_id': 'req-717ab50a-e86d-4849-9978-fa71a968f90e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:02.117257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] The token cache is being invalidate because project dc0aa7206a704d45a610dccc24ec8767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:02.117542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:02.118176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-717ab50a-e86d-4849-9978-fa71a968f90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:02.119731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1299/2602] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:01 2022] DELETE /identity/v3/projects/dc0aa7206a704d45a610dccc24ec8767 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:02.124665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.124955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.125233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.143296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:02.144430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:02.203092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1304/2603] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:02.210583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.211010 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.211290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.229212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:02.230395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:02.238188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1300/2604] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:02.243098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.252530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7882fa-7a89-4ba8-8caa-b61f8e9ec686 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.257015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.257655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.257935 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.258209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.259144 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:02.259452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.261034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.272291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1305/2605] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:02.276434 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.285637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde652bb-91cb-4071-a728-2784468cd0f0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.290785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.291429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.291708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.291982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.292704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.294573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.298488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:02.323749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1301/2606] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:02 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:02.329468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.340571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088314-ae5b-44f8-b62d-1df8c99d1ba5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.384101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:02.397673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.398270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.398551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.398881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.399567 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.401482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.420815 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1306/2607] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:02 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:02.424898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.436177 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141737c7-77b7-4ba8-a340-e16e5b554dc0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.440706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.441352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.441632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.441904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.442622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.444271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.453875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1302/2608] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:02.457521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.467200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5a368-9905-4225-a135-ed7a148acec8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.471745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.472385 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.472665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.472938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.473650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.475990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a0eb9-5d27-4203-be7f-77765abc042c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.486086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1307/2609] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:02.490015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.500839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dccca7-4ee9-4b8d-9291-4d2b0aa3465d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.505577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.506447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.506731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.507038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] PATH_INFO: `/v3/projects/b5eec6aba3d04b44a8e57bc796f0d7df/users/36ec45f5a2b94747aea686d365575513/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.521364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5eec6aba3d04b44a8e57bc796f0d7df, user_id=36ec45f5a2b94747aea686d365575513, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.524108 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.536737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1303/2610] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:02 2022] PUT /identity/v3/projects/b5eec6aba3d04b44a8e57bc796f0d7df/users/36ec45f5a2b94747aea686d365575513/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:02.547838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.548133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.548413 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.575018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] MFA Rules not processed for user `36ec45f5a2b94747aea686d365575513`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:02.576218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:02.639667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1308/2611] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:02.651511 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:02.653103 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:02.724038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73e23e-ed35-44cd-8ff1-b886db9b7ae3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:02.730180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:02.730856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:02.731148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:02.731430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:02.739213 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:02.740835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a0c01-d36d-49a6-b59e-db1d4923edac service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:02.749622 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1304/2612] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:05.398431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:05.456061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70545c9e-eaed-4c28-a78c-5ef0284d9ed2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:05.461236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:05.462104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:05.462388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:05.462665 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] PATH_INFO: `/v3/users/36ec45f5a2b94747aea686d365575513` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:05.463925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ec45f5a2b94747aea686d365575513)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:05.465804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:05.510212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ec45f5a2b94747aea686d365575513'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:05.517322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ec45f5a2b94747aea686d365575513'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:05.517968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ec45f5a2b94747aea686d365575513'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:05.522907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:05.531991 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ec45f5a2b94747aea686d365575513'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:05.532914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ec45f5a2b94747aea686d365575513'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:05.545708 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1309/2613] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:05 2022] DELETE /identity/v3/users/36ec45f5a2b94747aea686d365575513 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:05.550153 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:05.550670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:05.550793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:05.568004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:05.569821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:05.626203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1305/2614] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:19:05.636212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:05.637632 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:05.693743 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102e6bc-3d60-4d7f-a20c-7add1890a924 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:05.698056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:05.698693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:05.699011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:05.699293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:05.708338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:05.709795 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03db465-60dd-40bd-9310-c089903e9fed service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:05.719331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1310/2615] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:05.978300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:05.979818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:05.994578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8bd9b-143b-4698-9021-e544c2a780d0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:05.999414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.000954 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.001252 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.001536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] PATH_INFO: `/v3/projects/b5eec6aba3d04b44a8e57bc796f0d7df` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.002742 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5eec6aba3d04b44a8e57bc796f0d7df)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.004692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.039529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:06.054302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5eec6aba3d04b44a8e57bc796f0d7df', 'request_id': 'req-b3d783f8-d7da-4389-88d3-45e9268dd0a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:06.061813 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5eec6aba3d04b44a8e57bc796f0d7df', 'request_id': 'req-b3d783f8-d7da-4389-88d3-45e9268dd0a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:06.067952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5eec6aba3d04b44a8e57bc796f0d7df', 'request_id': 'req-b3d783f8-d7da-4389-88d3-45e9268dd0a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:06.078117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] The token cache is being invalidate because project b5eec6aba3d04b44a8e57bc796f0d7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:06.078409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:06.079056 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3d783f8-d7da-4389-88d3-45e9268dd0a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:06.080577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1306/2616] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:05 2022] DELETE /identity/v3/projects/b5eec6aba3d04b44a8e57bc796f0d7df => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:06.085779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.086092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.086425 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.104479 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:06.105671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:06.161971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1311/2617] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:06.170091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.170386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.170698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.190705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:06.191949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:06.200030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1307/2618] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:06.204091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.213956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7088-6630-4a41-966f-ffe83d1b9a8e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.218553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.219239 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.219525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.219803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.220524 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:06.220881 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.222415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.234563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1312/2619] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:06.238909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.248858 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bb0cf-677d-43c9-acbc-9809e8a2116a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.253460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.254102 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.254383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.254690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.255411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.257286 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.281462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1308/2620] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:06 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:06.285473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.295968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007d0bb-587b-47ae-824b-2cf16335abe0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.352512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.353182 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.353561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.353848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.354546 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.356495 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.365960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:06.378973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1313/2621] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:06 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:06.382835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.393236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57112e14-2070-44c6-8051-0435b9562b84 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.398345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.399036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.399320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.399597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.400350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.401932 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.412018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1309/2622] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:06.416487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.426254 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d9290-2395-4216-9bd9-ee3af38130ae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.430969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.431623 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.431908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.432189 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.432914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.434761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0ff1b-f3f6-4dd5-8bde-0cb7abdb37f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.444472 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1314/2623] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:06.448392 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.458104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d97e6-3d95-462f-8265-7c17a0ed6269 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.463345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.464204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.464486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.464791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] PATH_INFO: `/v3/projects/6cc1b75d801748ac8fb3cda7ce2f461c/users/0b885644c1ac4a8cad3c2eb61928ea98/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.479776 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1b75d801748ac8fb3cda7ce2f461c, user_id=0b885644c1ac4a8cad3c2eb61928ea98, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.482476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.496037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1310/2624] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:06 2022] PUT /identity/v3/projects/6cc1b75d801748ac8fb3cda7ce2f461c/users/0b885644c1ac4a8cad3c2eb61928ea98/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:06.507978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422b5f-37ac-4136-a930-560621604d8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.508274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422b5f-37ac-4136-a930-560621604d8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.508557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60422b5f-37ac-4136-a930-560621604d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.534208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-60422b5f-37ac-4136-a930-560621604d8d None None] MFA Rules not processed for user `0b885644c1ac4a8cad3c2eb61928ea98`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:06.535476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60422b5f-37ac-4136-a930-560621604d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:06.599165 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1315/2625] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:06.610128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.611892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:06.685830 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0538271-5797-42a5-858f-afd759c0bb67 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.691428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.692068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.692354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.692634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.700984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.702549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b02161-8385-4b97-bad8-b6898f99eea3 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:06.711452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1311/2626] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:06.917698 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60422b5f-37ac-4136-a930-560621604d8d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:06.977849 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60422b5f-37ac-4136-a930-560621604d8d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:06.982332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:06.983224 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:06.983506 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:06.983779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] PATH_INFO: `/v3/users/0b885644c1ac4a8cad3c2eb61928ea98` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:06.984982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b885644c1ac4a8cad3c2eb61928ea98)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:06.986947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.031976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b885644c1ac4a8cad3c2eb61928ea98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.039440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b885644c1ac4a8cad3c2eb61928ea98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.040072 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b885644c1ac4a8cad3c2eb61928ea98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.051889 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b885644c1ac4a8cad3c2eb61928ea98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.052516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b885644c1ac4a8cad3c2eb61928ea98'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.057604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:07.069919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1316/2627] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:06 2022] DELETE /identity/v3/users/0b885644c1ac4a8cad3c2eb61928ea98 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:07.074507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.074827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.075115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.092294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:07.093673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:07.130506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:07.153459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1312/2628] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:07.163527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.164954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:07.220730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2c5-7b3e-4ec3-9d6e-81fc61c3cc35 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.224398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.225038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.225949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.226230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.235170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.236605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce07207-16b2-4671-b75e-310fd0190036 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.246480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1317/2629] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:07.511691 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.513136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:07.526598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab4c3-68af-4fce-a611-df47e4850a99 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.531340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.532745 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.533035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.533315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] PATH_INFO: `/v3/projects/6cc1b75d801748ac8fb3cda7ce2f461c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.534693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc1b75d801748ac8fb3cda7ce2f461c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.536692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.583909 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc1b75d801748ac8fb3cda7ce2f461c', 'request_id': 'req-a6713aba-a071-49e5-bb96-78708a7d0737'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.590386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc1b75d801748ac8fb3cda7ce2f461c', 'request_id': 'req-a6713aba-a071-49e5-bb96-78708a7d0737'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.596348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc1b75d801748ac8fb3cda7ce2f461c', 'request_id': 'req-a6713aba-a071-49e5-bb96-78708a7d0737'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.607246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] The token cache is being invalidate because project 6cc1b75d801748ac8fb3cda7ce2f461c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:07.607529 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.608154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a6713aba-a071-49e5-bb96-78708a7d0737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:07.609669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1313/2630] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:07 2022] DELETE /identity/v3/projects/6cc1b75d801748ac8fb3cda7ce2f461c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:07.614516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.615035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.615160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.634689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:07.635933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:07.693086 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1318/2631] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:07.700923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.701426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.701536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.719407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:07.720610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:07.728989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1314/2632] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:07.733045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.743176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a416d-6e98-4c4c-9790-8b3f5a304468 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.747654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.748323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.748605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.748876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.749589 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:07.749890 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.751532 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.762737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1319/2633] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:07.766869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.776210 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf4252-5f92-46cd-b48c-ed3ad57adafb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.780927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.781565 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.781846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.782120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.782839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.784817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.810947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1315/2634] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:07 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:19:07.814945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.824844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0aa8f-2626-4ec7-b8cc-ec00cc15513b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.879376 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.880051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.880475 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.880753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.881530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.883788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.903573 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1320/2635] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:07 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:07.907563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.918373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5814d9a1-6b84-4472-8584-a5073a1d5c02 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.922989 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.923638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.923924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.924202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.924950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.926536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.936916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1316/2636] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:07.940652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.950348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc172e2-f469-4276-9562-ba642ede7d30 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.954874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.955562 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.955788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.956058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:07.956776 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:07.958338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ac82-9816-44d8-9c10-563da807d1b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:07.968460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1321/2637] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:19:07.972294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:07.982852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548594b-0cd5-442c-a7f9-b9fa1b0c9b7a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:07.987443 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:07.988303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:07.988584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:07.988857 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] PATH_INFO: `/v3/projects/8fa480aace4c440da1a0fd460f3de24c/users/34f066d5e9f14528ba294685c63df7ec/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.003299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa480aace4c440da1a0fd460f3de24c, user_id=34f066d5e9f14528ba294685c63df7ec, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.005905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.018133 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1317/2638] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:07 2022] PUT /identity/v3/projects/8fa480aace4c440da1a0fd460f3de24c/users/34f066d5e9f14528ba294685c63df7ec/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:08.028783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.029075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.029355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.035047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:08.057205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] MFA Rules not processed for user `34f066d5e9f14528ba294685c63df7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:08.058917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:08.120773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1322/2639] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:08.125181 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.185688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d66dd-3671-4b2b-aa7c-5d0977ec3f47 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.190260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.190947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.191231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.191510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.192203 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.194635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.198804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:08.222008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1318/2640] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:08 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:08.226566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.237153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077e6f8-912b-448b-ace5-59bea39c8dd5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.291606 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.292358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.292672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.292948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.293626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.295559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.315473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1323/2641] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:08 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:08.319366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.329583 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28393207-3c15-417e-91c0-7ee5ceaa3663 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.334196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.334872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.335160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.335441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.336159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.337748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.347969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1319/2642] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:08.352314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.361949 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac426b7-b97b-43bb-a37c-055e0cbb75ce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.367140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.368003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.368280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.368551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] PATH_INFO: `/v3/projects/4305203cd5a848058de7f8aa9a5adf59/users/1c011ce24d094cc9801eda02e53aa1c9/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.382638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4305203cd5a848058de7f8aa9a5adf59, user_id=1c011ce24d094cc9801eda02e53aa1c9, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.385308 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.398854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1324/2643] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:08 2022] PUT /identity/v3/projects/4305203cd5a848058de7f8aa9a5adf59/users/1c011ce24d094cc9801eda02e53aa1c9/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:08.402575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.412391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295c363-92ab-4207-98bd-51cf601a4b09 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.467872 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.468461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.468746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.469023 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.469739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.471348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40c78-6241-4980-92f4-9090b52801cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.481173 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1320/2644] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:08.485172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.495964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c270ef5-c235-44c7-a571-3f783ff16c08 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.500604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.501374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.501657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.501973 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] PATH_INFO: `/v3/projects/4305203cd5a848058de7f8aa9a5adf59/users/1c011ce24d094cc9801eda02e53aa1c9/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.503955 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4305203cd5a848058de7f8aa9a5adf59, user_id=1c011ce24d094cc9801eda02e53aa1c9, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.506550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.516673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1325/2645] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:08 2022] PUT /identity/v3/projects/4305203cd5a848058de7f8aa9a5adf59/users/1c011ce24d094cc9801eda02e53aa1c9/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:08.527548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.527908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.528281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.554785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] MFA Rules not processed for user `1c011ce24d094cc9801eda02e53aa1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:08.556005 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:08.623126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1321/2646] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:08.638001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.639730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:08.685902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:08.697198 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04d56-bb0f-450e-850c-352a5bf0bf74 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.700902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.701529 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.701810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.702084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.711389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.712803 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.722061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1326/2647] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:08.792596 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:08.802091 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31bd9e-1b9e-4f22-bcb2-bd5284e52700 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:08.805679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:08.806308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:08.806589 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:08.806901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:08.860212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:08.862609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:08.864372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c971db-85cd-41ab-a383-72581e7138bf service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:08.874385 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1322/2648] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:18.718061 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:18.871100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:19.643566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:19.704139 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c46c78-2513-4c8d-b3ab-ed5f216ba79d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:19.708907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:19.709824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:19.710102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:19.710400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] PATH_INFO: `/v3/users/34f066d5e9f14528ba294685c63df7ec` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:19.712222 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f066d5e9f14528ba294685c63df7ec)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:19.714117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:19.756689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f066d5e9f14528ba294685c63df7ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:19.763990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f066d5e9f14528ba294685c63df7ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:19.764621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f066d5e9f14528ba294685c63df7ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:19.776860 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f066d5e9f14528ba294685c63df7ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:19.777484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f066d5e9f14528ba294685c63df7ec'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:19.789293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1327/2649] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:19 2022] DELETE /identity/v3/users/34f066d5e9f14528ba294685c63df7ec => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:19.793739 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:19.794032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:19.794316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:19.812037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:19.813229 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:19.869447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1323/2650] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:19.878911 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:19.880359 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:19.938161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfb433-4fb1-48f0-9ba3-b6abb043d9dd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:19.943184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:19.943839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:19.944127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:19.944412 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:19.954796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:19.956231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:19.966083 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1328/2651] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:20.226380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:20.227961 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:20.245292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff0415-1281-48b6-844c-c74624c3c893 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:20.250109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:20.253120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:20.253120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:20.253120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] PATH_INFO: `/v3/projects/8fa480aace4c440da1a0fd460f3de24c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:20.253120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa480aace4c440da1a0fd460f3de24c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:20.254673 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:20.302991 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa480aace4c440da1a0fd460f3de24c', 'request_id': 'req-5b320603-df36-4d0b-8cc0-fa6074c874de'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.310905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa480aace4c440da1a0fd460f3de24c', 'request_id': 'req-5b320603-df36-4d0b-8cc0-fa6074c874de'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.316959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa480aace4c440da1a0fd460f3de24c', 'request_id': 'req-5b320603-df36-4d0b-8cc0-fa6074c874de'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.327792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] The token cache is being invalidate because project 8fa480aace4c440da1a0fd460f3de24c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:20.328082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.328737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.330292 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1324/2652] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:20 2022] DELETE /identity/v3/projects/8fa480aace4c440da1a0fd460f3de24c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:20.334200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:20.344402 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dba52-1da8-4bbc-a4c4-7d1ba42a4bf3 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:20.361749 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:20.402351 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:20.403450 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:20.403785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:20.404060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] PATH_INFO: `/v3/users/1c011ce24d094cc9801eda02e53aa1c9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:20.405227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c011ce24d094cc9801eda02e53aa1c9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:20.407175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:20.452831 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c011ce24d094cc9801eda02e53aa1c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.459657 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c011ce24d094cc9801eda02e53aa1c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.460268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c011ce24d094cc9801eda02e53aa1c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.472490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c011ce24d094cc9801eda02e53aa1c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.473121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-55074091-8e6a-43ca-97ec-06aa7ec80564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c011ce24d094cc9801eda02e53aa1c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.485162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1329/2653] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:20 2022] DELETE /identity/v3/users/1c011ce24d094cc9801eda02e53aa1c9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:20.757969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:20.759493 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:20.774333 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b320603-df36-4d0b-8cc0-fa6074c874de admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:20.831993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:20.832836 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:20.833115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:20.833387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] PATH_INFO: `/v3/projects/4305203cd5a848058de7f8aa9a5adf59` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:20.834545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4305203cd5a848058de7f8aa9a5adf59)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:20.836995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:20.866624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:20.884676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4305203cd5a848058de7f8aa9a5adf59', 'request_id': 'req-0c517af3-df7a-4de2-97b0-05e3a0572fc4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.891549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4305203cd5a848058de7f8aa9a5adf59', 'request_id': 'req-0c517af3-df7a-4de2-97b0-05e3a0572fc4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.897512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4305203cd5a848058de7f8aa9a5adf59', 'request_id': 'req-0c517af3-df7a-4de2-97b0-05e3a0572fc4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.908300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] The token cache is being invalidate because project 4305203cd5a848058de7f8aa9a5adf59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:20.908587 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.909215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0c517af3-df7a-4de2-97b0-05e3a0572fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:20.910727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1325/2654] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:20 2022] DELETE /identity/v3/projects/4305203cd5a848058de7f8aa9a5adf59 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:20.915656 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:20.916130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:20.916283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:20.934060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:20.935278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:20.992637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1330/2655] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:21.000353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.000647 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.000922 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.019547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:21.020733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:21.029261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1326/2656] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:19:21.033294 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.042917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127af67-d068-4054-8e6f-2adbcfedc977 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.047441 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.048087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.048373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.048682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.049366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:21.049671 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.051363 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.063334 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1331/2657] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:21.067532 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.077315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e60741-c472-402b-9862-11c5e4372d97 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.081950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.082582 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.082887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.083162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.083838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.086301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.110603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1327/2658] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/projects => generated 357 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:21.114553 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.124380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063188e9-29da-4503-b69a-4fa1f784d080 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.150390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:21.182370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.183050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.183335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.183877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.184551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.186461 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.206622 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1332/2659] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:21.211492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.221785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd289426-5121-4e52-8264-cd0d51ab92ae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.226391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.227083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.227369 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.227646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.228367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.229952 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.240089 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1328/2660] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:21.244053 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.253862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba49eab-2ef2-4c7d-bd67-eb61a867e1fe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.258976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.259621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.259903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.260177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.260896 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.262469 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f093d-ea8c-479f-bd9d-67a1303c1e4d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.272207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1333/2661] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:21.276853 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.286642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466b053-e908-4604-9425-49b13f0a3464 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.291460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.292320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.292705 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.292995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] PATH_INFO: `/v3/projects/d1f6afe1333d450d82345bda8dced8e8/users/d805fc400b184f23ae8275b0aec33ad0/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.308047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f6afe1333d450d82345bda8dced8e8, user_id=d805fc400b184f23ae8275b0aec33ad0, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.311798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.325043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1329/2662] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:21 2022] PUT /identity/v3/projects/d1f6afe1333d450d82345bda8dced8e8/users/d805fc400b184f23ae8275b0aec33ad0/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:21.335439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.335750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.336030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.361678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] MFA Rules not processed for user `d805fc400b184f23ae8275b0aec33ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:21.362943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:21.426481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1334/2663] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:21.431431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.491481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46eb4f-86ed-4a92-aba3-0003d6038739 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.496624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.497267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.497547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.497819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.498494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.500500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.525041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1330/2664] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/projects => generated 355 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:21.528993 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.539580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1f7d-9d1a-4caf-9ec0-e9e908e71d09 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.595610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.596274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.596559 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.596840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.597518 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.599511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.619181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1335/2665] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:21.623100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.633881 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab0442-3e0a-48c5-b798-ff9103673c64 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.638432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.639145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.639428 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.639699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.640414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.641983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.653250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1331/2666] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:21.657162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.667548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b1e13-46c3-4400-bc65-8dff8c62a968 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.672085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.672946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.673226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.673497 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] PATH_INFO: `/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8/users/f982f51cb40e42a8b9069305b96db6a7/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.688565 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb24012328bc4ff0a52ccc6c7d129af8, user_id=f982f51cb40e42a8b9069305b96db6a7, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.691238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.703938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1336/2667] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:21 2022] PUT /identity/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8/users/f982f51cb40e42a8b9069305b96db6a7/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:21.707785 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.717599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1faa4a-3b93-463e-b1df-ed2b420872fb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.737147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:21.773574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.774222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.774502 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.774798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.775515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.777121 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46201-3a3f-45f3-94d1-cb45c4f498a9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.786429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1332/2668] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:21.790971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.796451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:21.802554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f222-0ec2-4308-b7fb-b9881a15b3fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:21.808971 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:21.809868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.810147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.810421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] PATH_INFO: `/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8/users/f982f51cb40e42a8b9069305b96db6a7/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.812404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb24012328bc4ff0a52ccc6c7d129af8, user_id=f982f51cb40e42a8b9069305b96db6a7, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:21.815031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:21.825035 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1337/2669] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:21 2022] PUT /identity/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8/users/f982f51cb40e42a8b9069305b96db6a7/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:21.835869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:21.836161 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:21.836440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:21.862400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] MFA Rules not processed for user `f982f51cb40e42a8b9069305b96db6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:21.863686 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:21.929958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1333/2670] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:21 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:21.941440 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:21.943104 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:21.998611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bbcc6-636b-4314-b7a2-6dc9cc70027b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:22.002236 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:22.002899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:22.003179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:22.003453 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:22.012322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:22.013734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:22.023462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1338/2671] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:21 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:22.094615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:22.105257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3b308-d4b8-4b09-888a-c72ecff1049d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:22.108965 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:22.109664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:22.109949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:22.110271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:22.165617 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:22.167060 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd857782-3071-49e1-961b-1aef0e1ca471 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:22.175760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1334/2672] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:22 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:32.018821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:32.171778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=80 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:36.757394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:36.818052 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d14b-6244-4bca-97a1-c0398f601960 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:36.822602 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:36.823487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:36.823763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:36.824030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] PATH_INFO: `/v3/users/d805fc400b184f23ae8275b0aec33ad0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:36.825196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d805fc400b184f23ae8275b0aec33ad0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:36.827137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:36.871262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd805fc400b184f23ae8275b0aec33ad0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:36.878093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd805fc400b184f23ae8275b0aec33ad0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:36.878704 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd805fc400b184f23ae8275b0aec33ad0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:36.890404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd805fc400b184f23ae8275b0aec33ad0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:36.891062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd805fc400b184f23ae8275b0aec33ad0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:36.902941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1339/2673] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:36 2022] DELETE /identity/v3/users/d805fc400b184f23ae8275b0aec33ad0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:36.907375 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:36.907868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:36.907990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:36.925370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:36.926627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:36.992626 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1335/2674] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:37.002116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:37.003522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:37.059641 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa33bc-7b5e-4a7a-b9ea-7de6de3fee29 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:37.064273 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:37.064921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:37.065200 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:37.065473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:37.074621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:37.076079 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:37.085840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1340/2675] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:37.346338 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:37.347812 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:37.360545 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e257f-408a-461e-8905-e95c5cfc696c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:37.365116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:37.365972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:37.366255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:37.366535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] PATH_INFO: `/v3/projects/d1f6afe1333d450d82345bda8dced8e8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:37.367753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f6afe1333d450d82345bda8dced8e8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:37.369664 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:37.419191 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f6afe1333d450d82345bda8dced8e8', 'request_id': 'req-49570e73-260d-4c28-bfcf-464ea3bfbeba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.425982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f6afe1333d450d82345bda8dced8e8', 'request_id': 'req-49570e73-260d-4c28-bfcf-464ea3bfbeba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.432105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f6afe1333d450d82345bda8dced8e8', 'request_id': 'req-49570e73-260d-4c28-bfcf-464ea3bfbeba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.443159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] The token cache is being invalidate because project d1f6afe1333d450d82345bda8dced8e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:37.443474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.444111 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.446261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1336/2676] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:37 2022] DELETE /identity/v3/projects/d1f6afe1333d450d82345bda8dced8e8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:37.450378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:37.460575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edcb17-e636-4bff-897e-24b8c575a617 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:37.478022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:37.516789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:37.517587 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:37.517870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:37.518153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] PATH_INFO: `/v3/users/f982f51cb40e42a8b9069305b96db6a7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:37.519377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f982f51cb40e42a8b9069305b96db6a7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:37.521295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:37.565358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f982f51cb40e42a8b9069305b96db6a7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.573245 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f982f51cb40e42a8b9069305b96db6a7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.573859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f982f51cb40e42a8b9069305b96db6a7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.586150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f982f51cb40e42a8b9069305b96db6a7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.586783 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d00eefe2-7bcb-449c-b1b8-de5a5fea25c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f982f51cb40e42a8b9069305b96db6a7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:37.598644 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1341/2677] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:37 2022] DELETE /identity/v3/users/f982f51cb40e42a8b9069305b96db6a7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:37.866910 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:37.868503 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:37.881526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49570e73-260d-4c28-bfcf-464ea3bfbeba admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:37.938629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:37.939515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:37.939798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:37.940074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] PATH_INFO: `/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:37.941228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb24012328bc4ff0a52ccc6c7d129af8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:37.943156 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:37.971765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:37.992986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb24012328bc4ff0a52ccc6c7d129af8', 'request_id': 'req-ac0831ee-aa03-47d4-96bf-f65fb34af253'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.000045 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb24012328bc4ff0a52ccc6c7d129af8', 'request_id': 'req-ac0831ee-aa03-47d4-96bf-f65fb34af253'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.006061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb24012328bc4ff0a52ccc6c7d129af8', 'request_id': 'req-ac0831ee-aa03-47d4-96bf-f65fb34af253'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.016652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] The token cache is being invalidate because project cb24012328bc4ff0a52ccc6c7d129af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:38.016930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.017548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac0831ee-aa03-47d4-96bf-f65fb34af253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.019118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1337/2678] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:37 2022] DELETE /identity/v3/projects/cb24012328bc4ff0a52ccc6c7d129af8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:38.023976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.024445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.024552 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.042211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:38.043405 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.103684 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1342/2679] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:19:38.111731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.112027 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.112312 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.130946 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:38.132735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.141047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1338/2680] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:38.145310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.155175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97781d-c966-4dcc-a92e-18fe42ddbb12 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.159720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.160358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.160638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.160912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.161628 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:38.161932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.164102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.175485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1343/2681] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:38.180236 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.189481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb682fa-74e2-4f39-a33d-bfed5087789e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.194143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.194803 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.195094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.195367 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.196047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.197935 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.222815 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1339/2682] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/projects => generated 359 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:19:38.227414 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.237149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e5024-de20-4ba7-b573-ef3aa6d34cad admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.262329 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:38.294694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.295322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.295610 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.295888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.296582 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.298502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.317948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1344/2683] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/users => generated 280 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:38.321923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.332183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b9c71-f699-4c44-a13a-2c468b6880fe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.337011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.337668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.337953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.338232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.338992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.340613 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.351352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1340/2684] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:38.355159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.364948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01cc22-3757-4925-8da1-8fead7ce6947 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.370152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.370817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.371102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.371372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.372088 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.373654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81902-5deb-451f-948d-5c30a8cc349e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.383295 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1345/2685] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:38.387126 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.396675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06bfd4-a738-4917-9a48-a149e115c4f9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.401298 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.402155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.402437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.402711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] PATH_INFO: `/v3/projects/722ee4f0fc51441ab3258cd28b2ac18a/users/a3cfaed2be2945d3825da0a5f58eded9/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.417438 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722ee4f0fc51441ab3258cd28b2ac18a, user_id=a3cfaed2be2945d3825da0a5f58eded9, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.420119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.434304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1341/2686] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:38 2022] PUT /identity/v3/projects/722ee4f0fc51441ab3258cd28b2ac18a/users/a3cfaed2be2945d3825da0a5f58eded9/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:38.445310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.445598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.445872 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.471912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] MFA Rules not processed for user `a3cfaed2be2945d3825da0a5f58eded9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:38.473127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.536348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1346/2687] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:38.567071 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.568761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.625169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22928e6f-7eb7-4132-82b1-a26714ee4f31 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.628827 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.629460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.629743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.630019 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.638444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.639898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc6f76-b43b-419d-a4dd-5e386294e633 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.649351 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1342/2688] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:38.662798 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.724300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3fc4f-af14-4c13-afdc-ddaa4453dc59 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.729473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.730331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.730615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.730923 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] PATH_INFO: `/v3/users/a3cfaed2be2945d3825da0a5f58eded9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.732100 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cfaed2be2945d3825da0a5f58eded9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.734030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.774566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:38.780679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cfaed2be2945d3825da0a5f58eded9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.788981 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cfaed2be2945d3825da0a5f58eded9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.789624 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cfaed2be2945d3825da0a5f58eded9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.801431 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cfaed2be2945d3825da0a5f58eded9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.802023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cfaed2be2945d3825da0a5f58eded9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:38.814213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1347/2689] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:38 2022] DELETE /identity/v3/users/a3cfaed2be2945d3825da0a5f58eded9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:38.818674 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.819011 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.819295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.836631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:38.837843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.883537 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:38.897401 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1343/2690] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:38.907327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:38.908740 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:38.965231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6167a24-e645-48b0-b6c8-b0beed702a97 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:38.968899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:38.969541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:38.969826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:38.970107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:38.979147 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:38.980612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46800187-f1e2-4cc7-a575-5c109ff54392 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:38.989948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1348/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:39.255433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.257449 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:39.270267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaa8a9-c90a-433f-aec1-b5de005dc573 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.275599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.276451 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.276724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.276998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] PATH_INFO: `/v3/projects/722ee4f0fc51441ab3258cd28b2ac18a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.278184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] RBAC: Authorizing `identity:delete_project(project_id=722ee4f0fc51441ab3258cd28b2ac18a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.280162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.326202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722ee4f0fc51441ab3258cd28b2ac18a', 'request_id': 'req-3a50847f-b7c1-4245-8ab9-68f4c96a461d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:39.332666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722ee4f0fc51441ab3258cd28b2ac18a', 'request_id': 'req-3a50847f-b7c1-4245-8ab9-68f4c96a461d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:39.338886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722ee4f0fc51441ab3258cd28b2ac18a', 'request_id': 'req-3a50847f-b7c1-4245-8ab9-68f4c96a461d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:39.348861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] The token cache is being invalidate because project 722ee4f0fc51441ab3258cd28b2ac18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:39.349145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:39.349771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3a50847f-b7c1-4245-8ab9-68f4c96a461d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:39.351341 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1344/2692] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:39 2022] DELETE /identity/v3/projects/722ee4f0fc51441ab3258cd28b2ac18a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:39.356274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.356747 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.356845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.374745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:39.375972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:39.433769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1349/2693] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:39.441563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.441871 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.442154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.460763 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:39.461998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:39.469884 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1345/2694] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:39.474085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.483841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507efe3-8412-441a-95e4-13ad6f5b3783 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.488483 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.489170 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.489459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.489739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.490492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:39.490830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.492968 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.511908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1350/2695] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:39.516354 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.525744 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b30b9-2c92-4985-b442-b20f203d6da1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.530395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.531070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.531352 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.531623 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.532342 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.534268 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.559542 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1346/2696] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:19:39.563738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.574403 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4224f6-8798-4db2-a1e7-c61b1c136da7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.616723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:39.631304 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.632599 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.632878 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.633157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.633851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.635806 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.654888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1351/2697] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:19:39.659042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.669726 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6f856-256a-46be-b1cc-89f2ae5109f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.674311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.674985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.675274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.675547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.676287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.677851 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.688807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1347/2698] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:19:39.692482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.702184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d743e20-9079-4bb4-8010-70fdc9845578 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.707008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.707648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.707931 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.708208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.708919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.710494 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad091a4-5b39-419f-bda4-41c40e2f0af9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.720166 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1352/2699] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:39.724058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.734435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5579cc1-a57d-49c2-812e-03bff13792f3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.739117 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.739963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.740246 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.740524 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] PATH_INFO: `/v3/projects/8e46c75ebb294082af7f6607e9e205a6/users/16f2ff50b25a4251beba3da5de2a6e13/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.755258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e46c75ebb294082af7f6607e9e205a6, user_id=16f2ff50b25a4251beba3da5de2a6e13, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.757914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.770224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1348/2700] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:39 2022] PUT /identity/v3/projects/8e46c75ebb294082af7f6607e9e205a6/users/16f2ff50b25a4251beba3da5de2a6e13/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:39.780271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.780561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.780841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.808242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] MFA Rules not processed for user `16f2ff50b25a4251beba3da5de2a6e13`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:39.809720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:39.872393 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1353/2701] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:39.876814 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.937231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df80127-178d-4729-bf7b-41279981b66e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:39.941937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:39.942585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:39.942903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:39.943186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:39.943894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:39.945780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:39.963616 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:39.974866 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1349/2702] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:39.976171 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:39.986542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206074-1cf4-47ed-8146-3ad3a4109a83 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.045608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.046296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.046578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.046888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.047581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.049538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.071426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1354/2703] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:39 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:40.075196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.085892 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334e4c-2aeb-4160-8019-dd82d46dc0f3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.091413 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.092085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.092353 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.092636 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.093444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.095220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.107621 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1350/2704] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:40.111608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.121739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900556b1-5166-4287-b9f9-fc4aa0f46672 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.126579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.127225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.127482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.127755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.128476 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.130056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574496e-4f05-4b1b-9b4c-44a080318844 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.142082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1355/2705] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:40.146221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.157225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7069a6-1f1c-4dc3-a558-d491b82cdfe5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.161888 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.162806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.163651 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.163927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] PATH_INFO: `/v3/projects/9703c95225d4484ea58a143a3e741ff0/users/5ba76391abd34432ba72aa8830baf5f2/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.178792 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9703c95225d4484ea58a143a3e741ff0, user_id=5ba76391abd34432ba72aa8830baf5f2, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.181422 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.194423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1351/2706] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:40 2022] PUT /identity/v3/projects/9703c95225d4484ea58a143a3e741ff0/users/5ba76391abd34432ba72aa8830baf5f2/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:40.259484 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.259775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.260054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.291770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] MFA Rules not processed for user `5ba76391abd34432ba72aa8830baf5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:40.292989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:40.313105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:40.358110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1356/2707] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:40.362682 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.422152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056edcd9-7c15-4cd8-ae93-00ab770c34b3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.426791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.427446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.427730 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.428008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.428741 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.430630 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.455322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1352/2708] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:40 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:19:40.459409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.470430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681a158-12ae-4f88-ab9a-645d1610381f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.524714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.525380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.525700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.526051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.526868 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.528805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.548016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1357/2709] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:40 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:40.551987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.562388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffabbe-a91d-4f32-a441-818c524ec9cf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.567034 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.567683 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.567967 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.568244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.568962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.571107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.581308 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1353/2710] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:40.585829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.595674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1a77f-5419-472a-8688-755be105f37e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.600218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.601126 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.601404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.601677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] PATH_INFO: `/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340/users/be1b9333cc294642bebde07285576df5/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.615721 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cffa9a7231d4b20a5e8bbc51f849340, user_id=be1b9333cc294642bebde07285576df5, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.618373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.630943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1358/2711] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:40 2022] PUT /identity/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340/users/be1b9333cc294642bebde07285576df5/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:40.635294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.645234 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb7f71-9816-41f4-af12-f66b8ac0438f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.700178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.700778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.701061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.701336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.702048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.703666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d2f18-d0ab-4418-bc2c-eae91faec535 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.713586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1354/2712] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:40.717570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.727604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786838d-e458-47f1-a131-84f11e2e1804 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.732069 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.732938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.733428 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.733554 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] PATH_INFO: `/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340/users/be1b9333cc294642bebde07285576df5/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.735687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cffa9a7231d4b20a5e8bbc51f849340, user_id=be1b9333cc294642bebde07285576df5, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.738300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.748179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1359/2713] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:40 2022] PUT /identity/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340/users/be1b9333cc294642bebde07285576df5/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:40.758500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.758794 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.759086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.785835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] MFA Rules not processed for user `be1b9333cc294642bebde07285576df5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:40.787073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:40.807656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:40.855217 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1355/2714] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:40.866725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:40.868417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:40.924063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7c9e60-e371-4b8f-b69b-ed5b5908beac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:40.927648 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:40.928277 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:40.929087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:40.929370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:40.957750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:40.984430 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:40.985887 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:40.995146 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1360/2715] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:42.459953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:42.461409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:42.473276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e7536-ff9f-4c32-8809-fcb71d82e6f0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:42.476788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:42.477424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:42.477707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:42.477982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:42.532874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:42.534296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:42.543085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1356/2716] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:43.041696 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:43.050188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adfee1-f158-47c3-9aeb-b15268fc1057 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:43.053804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:43.054435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:43.054719 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:43.055027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:43.064763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:43.066177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c223b-e498-4e46-b731-b604a3e35e96 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:43.075706 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1361/2717] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:46.921054 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:47.003042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b485-44e4-488c-a609-7d1cdf3fbe5a service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:47.008669 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:47.009543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:47.009823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:47.011461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] PATH_INFO: `/v3/users/16f2ff50b25a4251beba3da5de2a6e13` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:47.011461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f2ff50b25a4251beba3da5de2a6e13)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:47.013239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:47.070718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f2ff50b25a4251beba3da5de2a6e13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.079825 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f2ff50b25a4251beba3da5de2a6e13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.080755 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f2ff50b25a4251beba3da5de2a6e13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.086030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:47.102079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f2ff50b25a4251beba3da5de2a6e13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.102869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f2ff50b25a4251beba3da5de2a6e13'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.120168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1357/2718] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:46 2022] DELETE /identity/v3/users/16f2ff50b25a4251beba3da5de2a6e13 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:47.123797 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:47.124679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:47.124972 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:47.145464 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:47.147485 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:47.152613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:47.154179 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:47.256800 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4415ee9-a6cf-43af-88eb-6004b916332c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:47.258733 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1362/2719] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:19:47.271690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:47.272528 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:47.272817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:47.273168 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:47.276215 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:47.276521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:47.278619 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:47.290294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:47.291955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:47.317687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1358/2720] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:47.356790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08792fb4-d6be-4305-a171-2f03402a6d2f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:47.360452 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:47.361103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:47.361388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:47.361668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:47.371439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:47.372879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:47.382327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1363/2721] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:47.885472 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:47.887178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:47.907317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c24a6-8359-4130-ba4d-7ad80797731d service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:47.911982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:47.912867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:47.913149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:47.913427 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] PATH_INFO: `/v3/projects/8e46c75ebb294082af7f6607e9e205a6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:47.914816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e46c75ebb294082af7f6607e9e205a6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:47.917242 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:47.968677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e46c75ebb294082af7f6607e9e205a6', 'request_id': 'req-c7159eb3-3f3f-4112-9650-c2f62a872785'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.978900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e46c75ebb294082af7f6607e9e205a6', 'request_id': 'req-c7159eb3-3f3f-4112-9650-c2f62a872785'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:47.985995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e46c75ebb294082af7f6607e9e205a6', 'request_id': 'req-c7159eb3-3f3f-4112-9650-c2f62a872785'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.000055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] The token cache is being invalidate because project 8e46c75ebb294082af7f6607e9e205a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:48.000055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.000055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.001553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1359/2722] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:47 2022] DELETE /identity/v3/projects/8e46c75ebb294082af7f6607e9e205a6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:48.005281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:48.017117 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373beaa-2620-4288-ae4c-d2fa33f5eeb4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:48.086291 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:48.087372 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:48.087766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:48.088286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] PATH_INFO: `/v3/users/5ba76391abd34432ba72aa8830baf5f2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:48.090142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba76391abd34432ba72aa8830baf5f2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:48.092180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:48.093946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:48.152152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba76391abd34432ba72aa8830baf5f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.160437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba76391abd34432ba72aa8830baf5f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.161192 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba76391abd34432ba72aa8830baf5f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.179454 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba76391abd34432ba72aa8830baf5f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.179975 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba76391abd34432ba72aa8830baf5f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.197526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1364/2723] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:48 2022] DELETE /identity/v3/users/5ba76391abd34432ba72aa8830baf5f2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:48.689220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:48.690748 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:48.705531 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7159eb3-3f3f-4112-9650-c2f62a872785 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:48.755862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:48.772248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:48.773104 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:48.773391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:48.773670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] PATH_INFO: `/v3/projects/9703c95225d4484ea58a143a3e741ff0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:48.774885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9703c95225d4484ea58a143a3e741ff0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:48.776790 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:48.823804 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9703c95225d4484ea58a143a3e741ff0', 'request_id': 'req-f19511f6-6c8b-461c-a969-f7f2897e8808'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.832063 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9703c95225d4484ea58a143a3e741ff0', 'request_id': 'req-f19511f6-6c8b-461c-a969-f7f2897e8808'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.838805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9703c95225d4484ea58a143a3e741ff0', 'request_id': 'req-f19511f6-6c8b-461c-a969-f7f2897e8808'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.849366 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] The token cache is being invalidate because project 9703c95225d4484ea58a143a3e741ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:48.849656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.850841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.852386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1360/2724] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:48 2022] DELETE /identity/v3/projects/9703c95225d4484ea58a143a3e741ff0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:48.856421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:48.857855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:48.871810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5c3b-2b8e-41a5-82eb-e2ef0a870276 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:48.930661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:48.931617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:48.931885 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:48.932159 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] PATH_INFO: `/v3/users/be1b9333cc294642bebde07285576df5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:48.933583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1b9333cc294642bebde07285576df5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:48.935548 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:48.979558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1b9333cc294642bebde07285576df5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.986520 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1b9333cc294642bebde07285576df5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.987209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1b9333cc294642bebde07285576df5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:48.999888 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1b9333cc294642bebde07285576df5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.000522 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ab639fe-3a11-4094-a1b9-df4691f66490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1b9333cc294642bebde07285576df5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.008062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:49.013435 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1365/2725] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:48 2022] DELETE /identity/v3/users/be1b9333cc294642bebde07285576df5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:49.290092 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.291713 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:49.305115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19511f6-6c8b-461c-a969-f7f2897e8808 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.362819 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.363668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.363953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.364231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] PATH_INFO: `/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.365407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cffa9a7231d4b20a5e8bbc51f849340)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.367340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.414008 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cffa9a7231d4b20a5e8bbc51f849340', 'request_id': 'req-1997f488-e107-4b5d-86e3-bd320d6cbba7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.421036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cffa9a7231d4b20a5e8bbc51f849340', 'request_id': 'req-1997f488-e107-4b5d-86e3-bd320d6cbba7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.427175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cffa9a7231d4b20a5e8bbc51f849340', 'request_id': 'req-1997f488-e107-4b5d-86e3-bd320d6cbba7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.437581 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] The token cache is being invalidate because project 8cffa9a7231d4b20a5e8bbc51f849340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:49.437869 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.438513 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1997f488-e107-4b5d-86e3-bd320d6cbba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:49.440048 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1361/2726] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:49 2022] DELETE /identity/v3/projects/8cffa9a7231d4b20a5e8bbc51f849340 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:49.445424 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.445962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.445962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.464755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:49.465964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:49.526206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1366/2727] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:49.534837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.535131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.535410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.542631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:49.555147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:49.556553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:49.564890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1362/2728] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:49.569682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.579650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab40472-a884-4687-90de-784ed523ee90 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.584322 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.585533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.585819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.586095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.586839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:49.587148 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.588753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.600400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1367/2729] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:49.604603 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.613915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa0173-a4c1-4fc9-9079-92af209d68e3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.618551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.619227 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.619514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.619791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.620488 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.622384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.648645 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1363/2730] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:49 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:49.652621 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.662500 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33b80-922e-47b7-a941-9f6f09147268 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.719149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.719805 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.720093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.720374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.721080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.723016 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.742384 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1368/2731] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:49 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:49.746448 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.757158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313ff9e-efba-4b42-b852-cbb1a6e6803e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.761782 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.762432 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.762723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.763039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.763768 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.765355 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.777335 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1364/2732] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:49.780969 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.790676 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a84593-4c78-40bc-b915-b2e6b7d7293b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.795261 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.795907 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.796193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.796470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.797190 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.798932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798e87c-cb57-4775-8e20-619019736a1e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.809537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1369/2733] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:19:49.813345 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.823796 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3ec96-72a1-435e-83cf-9215a134f3b8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:49.828553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:49.829454 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.829737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.830007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] PATH_INFO: `/v3/projects/9ecb841adc244aef80f6c3ece3677c9c/users/4f9f8d633d264b81a0566b9dab1c1ed4/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.844405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ecb841adc244aef80f6c3ece3677c9c, user_id=4f9f8d633d264b81a0566b9dab1c1ed4, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:49.847072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:49.859931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1365/2734] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:49 2022] PUT /identity/v3/projects/9ecb841adc244aef80f6c3ece3677c9c/users/4f9f8d633d264b81a0566b9dab1c1ed4/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:49.870612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac689-1def-404d-802e-5da404c12411 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:49.870936 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac689-1def-404d-802e-5da404c12411 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:49.871217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac689-1def-404d-802e-5da404c12411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:49.897129 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c44ac689-1def-404d-802e-5da404c12411 None None] MFA Rules not processed for user `4f9f8d633d264b81a0566b9dab1c1ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:49.898344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c44ac689-1def-404d-802e-5da404c12411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:49.955513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:49.966244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1370/2735] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:49.979116 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:49.981033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:50.042079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322238f-7e53-4e2a-b187-7ea3c37f86b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:50.046356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:50.047021 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:50.047304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:50.047577 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:50.056928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:50.058602 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:50.076170 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1366/2736] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:51.215661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac689-1def-404d-802e-5da404c12411 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:51.216631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c44ac689-1def-404d-802e-5da404c12411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:51.274407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac689-1def-404d-802e-5da404c12411 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:51.277989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:51.278625 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:51.278938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:51.279215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:51.280966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:51.342599 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:51.344116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:51.353829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1371/2737] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:54.457703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:54.459212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:54.473225 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54daddf-4ac0-4276-90f7-e8b774a23424 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:54.478599 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:54.479183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:54.479465 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:54.479743 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] PATH_INFO: `/v3/users/0e8bf7bbabc4432fa88002ea5c1dce4c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:54.480992 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8bf7bbabc4432fa88002ea5c1dce4c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:54.482953 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:54.529811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8bf7bbabc4432fa88002ea5c1dce4c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.536364 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8bf7bbabc4432fa88002ea5c1dce4c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.536994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8bf7bbabc4432fa88002ea5c1dce4c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.548575 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8bf7bbabc4432fa88002ea5c1dce4c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.549207 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a41f9fc0-575e-4135-99f4-7e72fdd8732f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8bf7bbabc4432fa88002ea5c1dce4c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.561908 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1367/2738] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:19:54 2022] DELETE /identity/v3/users/0e8bf7bbabc4432fa88002ea5c1dce4c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:54.730549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:54.740678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f9f54-72b2-4e9c-934d-a8ee27e3f95e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:54.796604 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:54.797386 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:54.797667 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:54.797939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] PATH_INFO: `/v3/projects/c0aec4ac096f4f30b7c357db043925fd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:54.799131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0aec4ac096f4f30b7c357db043925fd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:54.801023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:54.844679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:54.847737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0aec4ac096f4f30b7c357db043925fd', 'request_id': 'req-c12b5b79-65f0-4f99-afe4-10928b5b48fe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.858496 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0aec4ac096f4f30b7c357db043925fd', 'request_id': 'req-c12b5b79-65f0-4f99-afe4-10928b5b48fe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.868177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0aec4ac096f4f30b7c357db043925fd', 'request_id': 'req-c12b5b79-65f0-4f99-afe4-10928b5b48fe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.878400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] The token cache is being invalidate because project c0aec4ac096f4f30b7c357db043925fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:19:54.878674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.879310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c12b5b79-65f0-4f99-afe4-10928b5b48fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:19:54.880820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1372/2739] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:19:54 2022] DELETE /identity/v3/projects/c0aec4ac096f4f30b7c357db043925fd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:19:54.885740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:54.886030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:54.886307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:54.903737 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:54.904929 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:54.914570 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:54.964633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1368/2740] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:54.972218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:54.972508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:54.972787 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:54.991945 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:54.993142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:55.000943 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1373/2741] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:55.005002 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.014508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d477c6-1fcd-4043-8439-a79fdaeb4b90 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.019059 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.019698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.019976 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.020250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.020974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:19:55.021281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.023433 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.034727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1369/2742] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:19:55.039406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.048513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85152ad-712a-4b6f-bdfc-611b2a13046b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.053096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.053728 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.054008 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.054283 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.055003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.056880 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.083163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1374/2743] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:19:55 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:55.087852 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.097761 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298ebd9-99b7-4c41-b2a6-8fbbaf86a4d0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.152275 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.152957 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.153237 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.153512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.154201 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.156145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.176290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1370/2744] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:19:55 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:19:55.180547 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.191164 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6afc66-93f7-43c0-945d-0d5a934bbfcf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.196046 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.196713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.196997 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.197271 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.197994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.199661 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.211010 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1375/2745] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:55.214897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.224924 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb090c-aaa4-4bed-b35a-a8e98855e691 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.230140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.230817 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.231107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.231384 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.232110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.233697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d38682-a17b-4ecc-a21e-a76ba8d5dc96 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.243361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1371/2746] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:19:55.247627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.258001 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948499a-191c-43c7-8568-e50ecf78147f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.262789 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.263668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.263947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.264217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] PATH_INFO: `/v3/projects/a1c0ce09b9e34dd1a5dcce0a3e37fe31/users/a8b26629e86b43dfaacc4de87a26ebd4/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.279799 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c0ce09b9e34dd1a5dcce0a3e37fe31, user_id=a8b26629e86b43dfaacc4de87a26ebd4, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.282455 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.296013 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1376/2747] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:19:55 2022] PUT /identity/v3/projects/a1c0ce09b9e34dd1a5dcce0a3e37fe31/users/a8b26629e86b43dfaacc4de87a26ebd4/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:19:55.301249 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb852fa-2572-4991-ade3-8be9f5dc542d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.301559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb852fa-2572-4991-ade3-8be9f5dc542d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.301885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb852fa-2572-4991-ade3-8be9f5dc542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.320289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-deb852fa-2572-4991-ade3-8be9f5dc542d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:55.321501 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-deb852fa-2572-4991-ade3-8be9f5dc542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:55.380407 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1372/2748] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:19:55.390818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:55.392568 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:55.453152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4104f-9556-420f-a35c-9ad1dd9f6ca2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:55.456840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:55.457493 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:55.457912 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:55.458218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:55.469030 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:55.470540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:55.485877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1377/2749] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:19:59.527090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de0c6-39d9-4645-aebd-dae555ce19e4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:59.527558 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de0c6-39d9-4645-aebd-dae555ce19e4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:59.527752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676de0c6-39d9-4645-aebd-dae555ce19e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:59.555189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-676de0c6-39d9-4645-aebd-dae555ce19e4 None None] MFA Rules not processed for user `a8b26629e86b43dfaacc4de87a26ebd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:19:59.556348 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-676de0c6-39d9-4645-aebd-dae555ce19e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:59.565886 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:59.659293 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1373/2750] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:19:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:19:59.670924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:19:59.672348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:19:59.771056 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc46099-0722-4c6a-8f80-1327e0899bd0 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:19:59.780225 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:19:59.780932 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:19:59.781217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:19:59.781678 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:19:59.793791 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:19:59.795264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:19:59.799276 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:19:59.806670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1378/2751] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:19:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:01.120676 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:01.121165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:01.121265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:01.138653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] MFA Rules not processed for user `a8b26629e86b43dfaacc4de87a26ebd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:01.139823 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:01.146725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1374/2752] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:02.845205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:02.848854 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:02.932811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2065719-af94-4cbd-bf49-3bbc6d0d42e3 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:02.938437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:02.939290 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:02.939581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:02.939861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:02.942934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:02.955122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:02.956766 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3aa887-6a8c-474e-96fa-6aad53b2a970 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:02.969288 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1379/2753] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:11.133452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:12.964761 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=33 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:32.486901 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:32.488343 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:32.501514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdfef-a2ca-493e-9fbb-795ce663ddfa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:32.505937 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:32.506811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:32.507160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:32.507435 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] PATH_INFO: `/v3/users/4f9f8d633d264b81a0566b9dab1c1ed4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:32.508606 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9f8d633d264b81a0566b9dab1c1ed4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:32.510498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:32.556118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9f8d633d264b81a0566b9dab1c1ed4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:32.563765 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9f8d633d264b81a0566b9dab1c1ed4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:32.564419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9f8d633d264b81a0566b9dab1c1ed4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:32.576176 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9f8d633d264b81a0566b9dab1c1ed4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:32.576795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9f8d633d264b81a0566b9dab1c1ed4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:32.589155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1375/2754] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:20:32 2022] DELETE /identity/v3/users/4f9f8d633d264b81a0566b9dab1c1ed4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:20:32.593731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:32.594208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:32.594326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:32.611922 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:32.613113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:32.670557 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1380/2755] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:32.680079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:32.681473 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:32.738143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de21bb-9544-42f1-bb72-80efa57a0325 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:32.741689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:32.742318 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:32.742597 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:32.742905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:32.751724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:32.753135 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058385d-66a5-4294-9c81-c3b4e80cca13 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:32.762359 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1376/2756] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:33.020579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.022044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:33.034531 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3f81-42d7-414c-8e81-0838f2add57d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.039101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.039954 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.040231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.040504 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] PATH_INFO: `/v3/projects/9ecb841adc244aef80f6c3ece3677c9c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.041677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ecb841adc244aef80f6c3ece3677c9c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.043635 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.089898 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ecb841adc244aef80f6c3ece3677c9c', 'request_id': 'req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:33.095899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ecb841adc244aef80f6c3ece3677c9c', 'request_id': 'req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:33.103162 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ecb841adc244aef80f6c3ece3677c9c', 'request_id': 'req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:33.113658 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] The token cache is being invalidate because project 9ecb841adc244aef80f6c3ece3677c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:20:33.113946 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:33.114571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df6298a8-a38e-43e0-96cf-9fbbce75eb8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:33.116137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1381/2757] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:20:33 2022] DELETE /identity/v3/projects/9ecb841adc244aef80f6c3ece3677c9c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:20:33.121012 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.121299 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.121573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.139077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:33.140265 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:33.197940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1377/2758] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:33.206420 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.206709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.207042 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.226510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:33.227804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:33.235710 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1382/2759] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:20:33.239846 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.250070 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f576d6-6c59-4c16-b2d8-a2cfbe38334b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.254624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.255304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.255585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.255874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.256584 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:20:33.256890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.259124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.270337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1378/2760] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:20:33.274207 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.283315 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55e44-bf90-4fa9-bf91-f8f9548926fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.287871 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.288510 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.288796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.289074 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.289764 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.291701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.317381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1383/2761] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:20:33 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:20:33.321918 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.327155 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:33.333033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cfd83-0813-4f1c-989b-b0e10598cc2e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.389816 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.390481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.390781 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.391073 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.391749 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.393643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.412808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1379/2762] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:20:33 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:20:33.416926 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.427024 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501aad6-c583-450c-ab75-f77e34a8f1c2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.432128 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.432802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.433344 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.433647 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.434364 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.435990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.446754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1384/2763] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:20:33.450508 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.460261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a8061-e4a1-4923-b563-ba72a4f1397f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.464880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.465517 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.465799 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.466074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.466879 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.468452 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887c732-dcb9-44b1-b95d-a12337c029aa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.478024 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1380/2764] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:20:33.481903 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.492584 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40640353-3c42-4c41-85aa-9e0ead594fd1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.497286 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.498151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.498433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.498709 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] PATH_INFO: `/v3/projects/9b16942a1fe6467989aaaecbd4f178c4/users/1cb7a6e99e2b49bba421a7d7c9edabc7/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.513499 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b16942a1fe6467989aaaecbd4f178c4, user_id=1cb7a6e99e2b49bba421a7d7c9edabc7, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.516145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.528511 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1385/2765] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:20:33 2022] PUT /identity/v3/projects/9b16942a1fe6467989aaaecbd4f178c4/users/1cb7a6e99e2b49bba421a7d7c9edabc7/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:20:33.538638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ac4f5-4dd4-4b8d-b882-bea1804ef927 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.539134 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ac4f5-4dd4-4b8d-b882-bea1804ef927 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.539274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ac4f5-4dd4-4b8d-b882-bea1804ef927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.565183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-077ac4f5-4dd4-4b8d-b882-bea1804ef927 None None] MFA Rules not processed for user `1cb7a6e99e2b49bba421a7d7c9edabc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:33.566371 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-077ac4f5-4dd4-4b8d-b882-bea1804ef927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:33.629981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1381/2766] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:20:33.640765 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:33.642491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:33.664429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:33.700670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d99741-6fd1-4b50-b6f4-573616345bfe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:33.704268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:33.704904 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:33.705188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:33.705462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:33.713636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:33.715090 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:33.723838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1386/2767] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:20:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:35.056077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e2b2-9514-4b3b-85f2-a5241f50f66a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:35.056557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e2b2-9514-4b3b-85f2-a5241f50f66a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:35.056656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e2b2-9514-4b3b-85f2-a5241f50f66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:35.075563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1883e2b2-9514-4b3b-85f2-a5241f50f66a None None] MFA Rules not processed for user `1cb7a6e99e2b49bba421a7d7c9edabc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:35.076760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1883e2b2-9514-4b3b-85f2-a5241f50f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:35.083601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1382/2768] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:43.719773 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=17 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:45.069708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:45.266000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:45.267526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:45.348161 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04e3aa-e312-44d3-bec6-16bd3baa85be service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:45.354062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:45.354714 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:45.355028 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:45.355311 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:45.357093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:45.369611 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:45.371168 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:45.381570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1387/2769] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:20:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:52.349340 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a403ae-53b2-44f9-bdc9-fd58c6459285 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:52.349805 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a403ae-53b2-44f9-bdc9-fd58c6459285 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:52.349913 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a403ae-53b2-44f9-bdc9-fd58c6459285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:52.367326 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-89a403ae-53b2-44f9-bdc9-fd58c6459285 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:52.368494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-89a403ae-53b2-44f9-bdc9-fd58c6459285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:52.422143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1383/2770] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:20:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:52.468127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:52.469519 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:52.536746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668b0f2-84a5-4037-b098-5823bdd88404 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:52.541846 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:52.542491 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:52.542788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:52.543095 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:52.551196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:52.552772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:52.561082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1388/2771] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:20:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:20:53.046934 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4362173-02b9-4ba6-9dfc-6efef0b16ef7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.047301 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4362173-02b9-4ba6-9dfc-6efef0b16ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.047585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4362173-02b9-4ba6-9dfc-6efef0b16ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.065681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b4362173-02b9-4ba6-9dfc-6efef0b16ef7 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:53.066903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b4362173-02b9-4ba6-9dfc-6efef0b16ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:53.075412 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1384/2772] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:53.087321 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:53.097816 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485da257-455e-4a98-8bf0-8cfe20e98fed service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:53.103347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:53.103978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.104265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.104543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.112536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:53.114106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:53.122850 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1389/2773] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:20:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:20:53.308489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2be9-d575-461b-ae76-5a008d4b335a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.308780 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2be9-d575-461b-ae76-5a008d4b335a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.309079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e2be9-d575-461b-ae76-5a008d4b335a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.326309 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a05e2be9-d575-461b-ae76-5a008d4b335a None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:53.327534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a05e2be9-d575-461b-ae76-5a008d4b335a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:53.334555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1385/2774] 10.222.0.48 () {58 vars in 1009 bytes} [Fri Jul 1 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:53.345933 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:53.357326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa5611-fe6c-4d37-a31e-5283fb1ae854 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:53.362348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:53.363009 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.363297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.363579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.372150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:53.373730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:53.381717 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1390/2775] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:20:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:20:53.694703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.695222 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.695723 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.714706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] MFA Rules not processed for user `b73d15dbbe6d4bfc99ea858b7e12c3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:53.715959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:53.723082 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1386/2776] 10.222.0.48 () {58 vars in 967 bytes} [Fri Jul 1 07:20:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:53.735578 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:53.746739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765bf2-75d0-4a9a-8966-0c00c898bda1 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:53.751995 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eae0222c39364fa69be2a3cc938516bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:53.752634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:53.752920 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:53.753196 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:53.761830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:53.763459 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bfcf0-df01-47eb-a2d1-9971641ceba4 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:53.771437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1391/2777] 10.222.0.48 () {60 vars in 1384 bytes} [Fri Jul 1 07:20:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 01 07:20:55.791563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:55.853631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8fdc4-950d-47ef-af62-09a6fe65ef35 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:55.858128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:55.858999 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:55.859285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:55.859557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] PATH_INFO: `/v3/users/1cb7a6e99e2b49bba421a7d7c9edabc7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:55.860729 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cb7a6e99e2b49bba421a7d7c9edabc7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:55.862628 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:55.906521 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cb7a6e99e2b49bba421a7d7c9edabc7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:55.913962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cb7a6e99e2b49bba421a7d7c9edabc7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:55.914873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cb7a6e99e2b49bba421a7d7c9edabc7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:55.928330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cb7a6e99e2b49bba421a7d7c9edabc7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:55.928330 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cb7a6e99e2b49bba421a7d7c9edabc7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:55.946078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1387/2778] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:20:55 2022] DELETE /identity/v3/users/1cb7a6e99e2b49bba421a7d7c9edabc7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:20:55.950566 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:55.951110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:55.951212 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:55.968570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:20:55.969775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:55.999580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:56.028549 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1392/2779] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:20:56.038221 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:56.039649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:56.046100 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:20:56.098504 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a4b6c-7c81-4498-8583-3c4276610b75 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:56.102703 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:56.103370 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:56.103704 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:56.103933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:56.112882 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:56.114294 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:56.124356 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1388/2780] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:20:56.291617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:20:56.293163 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:20:56.306019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791acb78-1b35-47be-8e80-d5f2d009a612 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:20:56.311303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:20:56.312297 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:20:56.312592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:20:56.312982 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] PATH_INFO: `/v3/projects/9b16942a1fe6467989aaaecbd4f178c4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:20:56.314186 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b16942a1fe6467989aaaecbd4f178c4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:20:56.316195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:20:56.365140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b16942a1fe6467989aaaecbd4f178c4', 'request_id': 'req-e484f564-1cf2-4b7c-84e2-57a32914d05c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:56.371668 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b16942a1fe6467989aaaecbd4f178c4', 'request_id': 'req-e484f564-1cf2-4b7c-84e2-57a32914d05c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:56.378941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b16942a1fe6467989aaaecbd4f178c4', 'request_id': 'req-e484f564-1cf2-4b7c-84e2-57a32914d05c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:56.389301 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] The token cache is being invalidate because project 9b16942a1fe6467989aaaecbd4f178c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:20:56.389615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:56.390242 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:20:56.391811 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1393/2781] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:20:56 2022] DELETE /identity/v3/projects/9b16942a1fe6467989aaaecbd4f178c4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:21:06.120074 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=24 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:21:06.386229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:22:33.088637 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:22:33.090085 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:22:33.230300 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae7fc8-d419-4776-8d13-c7900d808052 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:22:33.287661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:22:33.288334 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:22:33.288614 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:22:33.288889 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:22:33.290642 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:22:33.302090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:22:33.303684 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:22:33.313588 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1389/2782] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:22:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:22:43.309474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:23:58.928341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:23:58.929947 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:23:59.009908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484f564-1cf2-4b7c-84e2-57a32914d05c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:23:59.013581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:23:59.014226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:23:59.014543 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:23:59.014853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:23:59.016629 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:23:59.103107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:23:59.104533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e8ae-e134-4c55-9157-4ba8122261fd service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:23:59.123830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1394/2783] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:23:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:24:00.297845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:24:00.307547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1883b2b-8900-4dfe-8f89-fa5f00d4549a service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:24:00.311067 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:24:00.311688 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:24:00.311972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:24:00.312243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:24:00.313950 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:24:00.327031 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:24:00.328440 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:24:00.339689 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1390/2784] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:24:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:24:04.657193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:24:04.657713 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:24:04.657820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:24:04.675358 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] MFA Rules not processed for user `a8b26629e86b43dfaacc4de87a26ebd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:24:04.677066 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:24:04.720867 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:24:04.731374 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1395/2785] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:24:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:24:05.027441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:24:05.035920 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b795-b990-4d4b-be8d-6ccd8e8b3132 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:24:05.039506 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:24:05.040128 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:24:05.040408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:24:05.040681 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:24:05.042410 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:24:05.051735 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:24:05.053136 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:24:05.061244 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1391/2786] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:24:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:24:11.160757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:24:11.171319 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e660d6-bcc2-4479-8c13-ca9c70699802 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:24:11.175099 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:24:11.176398 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:24:11.176683 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:24:11.176989 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:24:11.178925 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:24:11.189723 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:24:11.191262 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:24:11.200284 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1396/2787] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:24:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:24:15.057751 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=12 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:24:21.196211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:25:37.031250 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:25:37.032680 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:25:37.049180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a374dd-e145-4eb5-bb02-d02ed2a8ea27 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:25:37.055519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:25:37.056158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:25:37.056710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:25:37.056990 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:25:37.058806 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:25:37.069125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:25:37.070690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:25:37.079591 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1392/2788] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:25:47.075625 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:01.796133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:01.812230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15ac7c-0517-4f13-a74d-ffab8aee39f4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:01.817956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:01.818609 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:01.819003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:01.819231 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:01.821122 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:01.835516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:01.838073 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:01.856654 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1397/2789] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:06.563560 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:06.564956 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:06.577189 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac638b-645f-4111-b2e1-36da4b377295 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:06.580773 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:06.581408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:06.581696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:06.581977 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:06.583724 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:06.667484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:06.668930 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:06.688039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1393/2790] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:09.639312 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:09.640767 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:09.654513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e5498-fa83-49ad-89aa-dd1ecf038835 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:09.659542 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:09.660183 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:09.660468 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:09.660746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:09.662492 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:09.675616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:09.677314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:09.688536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1398/2791] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:09.704175 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:09.705610 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:09.720035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2432-f8c5-4eba-90cf-e9281d6a4431 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:09.724656 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:09.725518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:09.725798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:09.726069 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] PATH_INFO: `/v3/users/a8b26629e86b43dfaacc4de87a26ebd4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:09.727302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b26629e86b43dfaacc4de87a26ebd4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:09.729235 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:09.790408 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b26629e86b43dfaacc4de87a26ebd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:09.798722 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b26629e86b43dfaacc4de87a26ebd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:09.799515 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b26629e86b43dfaacc4de87a26ebd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:09.812734 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b26629e86b43dfaacc4de87a26ebd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:09.813409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c5de19b-8b0e-4b5c-9d38-f3670f41691e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b26629e86b43dfaacc4de87a26ebd4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:09.832615 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1394/2792] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:09 2022] DELETE /identity/v3/users/a8b26629e86b43dfaacc4de87a26ebd4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:10.022292 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.034282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346c61b-e354-41d9-b43b-068e0003026f service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.090278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.091167 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.091456 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.091734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] PATH_INFO: `/v3/projects/a1c0ce09b9e34dd1a5dcce0a3e37fe31` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.092929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c0ce09b9e34dd1a5dcce0a3e37fe31)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.094957 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.139753 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c0ce09b9e34dd1a5dcce0a3e37fe31', 'request_id': 'req-7acfe626-824a-49b2-a40f-e10b3d81062e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:10.146233 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c0ce09b9e34dd1a5dcce0a3e37fe31', 'request_id': 'req-7acfe626-824a-49b2-a40f-e10b3d81062e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:10.153217 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c0ce09b9e34dd1a5dcce0a3e37fe31', 'request_id': 'req-7acfe626-824a-49b2-a40f-e10b3d81062e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:10.164043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] The token cache is being invalidate because project a1c0ce09b9e34dd1a5dcce0a3e37fe31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:10.164335 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:10.164953 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7acfe626-824a-49b2-a40f-e10b3d81062e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:10.166462 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1399/2793] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:10 2022] DELETE /identity/v3/projects/a1c0ce09b9e34dd1a5dcce0a3e37fe31 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:10.171482 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.171778 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.172061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.228062 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:10.231166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:10.232955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:10.292534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1395/2794] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:10.300686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.300980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.301265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.319406 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:10.320617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:10.328563 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1400/2795] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:10.332995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.342419 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a51c-d41b-4c1f-b591-cc15b7e58918 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.347598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.348267 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.348550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.348818 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.349539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:26:10.349845 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.351484 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.362576 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1396/2796] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:26:10.366659 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.375689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa92b0-07f5-4db4-a9da-5e0db8ef00c4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.380331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.380967 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.381250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.381526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.382209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.384143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.409392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1401/2797] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:10 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:26:10.413839 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.423653 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69238a-02cd-425c-9fb2-25a8276b8798 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.479632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.480291 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.480578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.480859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.481550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.483509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.504258 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1397/2798] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:10 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:10.508327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.518513 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02626fe-00a8-42c1-b59a-d22c3cb1f552 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.523693 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.524347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.524634 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.524915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.525636 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.527280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.537985 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1402/2799] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:10.541609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.551386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51f4c-5b3b-44cc-a6e0-e3b7f6640857 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.556311 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.556970 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.557257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.557536 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.558260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.560131 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7be35e-ea74-43bf-b0e4-8d78c319d9a6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.570445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1398/2800] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:26:10.574418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.584999 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ed4a-7f23-4782-aafd-c4d66491d06c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.589741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.591011 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.591310 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.591593 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] PATH_INFO: `/v3/projects/d080b37c518947ee978a599a5c334a13/users/175ada94bc744ea09050b2869251f665/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.605837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d080b37c518947ee978a599a5c334a13, user_id=175ada94bc744ea09050b2869251f665, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.608527 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.616760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:10.621966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1403/2801] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:10 2022] PUT /identity/v3/projects/d080b37c518947ee978a599a5c334a13/users/175ada94bc744ea09050b2869251f665/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:10.632639 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.632933 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.633212 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.659571 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] MFA Rules not processed for user `175ada94bc744ea09050b2869251f665`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:10.660733 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:10.723926 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1399/2802] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:10.734508 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:10.736244 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:10.814118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17002c-47e2-44f7-8023-053672bdf124 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:10.818383 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:10.819059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:10.819356 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:10.819722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:10.828032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:10.829466 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be350bda-b8c6-4ae6-9708-6b8c1a890e03 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:10.838844 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1404/2803] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:10 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:19.237197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:19.241320 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:19.299243 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d81580-1c4e-4698-a710-d736b4449b16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:19.303795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:19.304655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:19.304936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:19.305228 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] PATH_INFO: `/v3/users/175ada94bc744ea09050b2869251f665` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:19.306633 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175ada94bc744ea09050b2869251f665)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:19.308553 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:19.354248 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175ada94bc744ea09050b2869251f665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.360697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175ada94bc744ea09050b2869251f665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.361415 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175ada94bc744ea09050b2869251f665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.373232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175ada94bc744ea09050b2869251f665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.373856 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175ada94bc744ea09050b2869251f665'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.385608 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1400/2804] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:19 2022] DELETE /identity/v3/users/175ada94bc744ea09050b2869251f665 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:19.390313 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:19.390859 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:19.390980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:19.408337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:19.409556 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:19.467032 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1405/2805] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:19.477047 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:19.478437 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:19.534877 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303d61d-cee8-498f-a6bd-fe5919532085 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:19.538519 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:19.539195 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:19.539481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:19.539760 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:19.548936 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:19.550361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c71fa9-232a-4de7-8a73-cbb8fd91d6a8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:19.559995 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1401/2806] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:19.825772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:19.827250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:19.840638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacdd02-4d88-4ca3-affb-b69f307ade8b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:19.845380 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:19.846317 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:19.846673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:19.847180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] PATH_INFO: `/v3/projects/d080b37c518947ee978a599a5c334a13` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:19.848770 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d080b37c518947ee978a599a5c334a13)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:19.851089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:19.898432 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd080b37c518947ee978a599a5c334a13', 'request_id': 'req-3f0f2e8e-797c-45b3-817b-3b025c5a857c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.904419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd080b37c518947ee978a599a5c334a13', 'request_id': 'req-3f0f2e8e-797c-45b3-817b-3b025c5a857c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.911530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd080b37c518947ee978a599a5c334a13', 'request_id': 'req-3f0f2e8e-797c-45b3-817b-3b025c5a857c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.921820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] The token cache is being invalidate because project d080b37c518947ee978a599a5c334a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:19.922109 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.922730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3f0f2e8e-797c-45b3-817b-3b025c5a857c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:19.924280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1406/2807] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:19 2022] DELETE /identity/v3/projects/d080b37c518947ee978a599a5c334a13 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:19.929791 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:19.930083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:19.930387 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:19.948696 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:19.949897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:19.968481 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:20.011442 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1402/2808] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:20.019269 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.019585 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.019886 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.033282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:20.039229 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:20.041278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:20.050065 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1407/2809] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:20.054670 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.064492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b5eb-b7b8-43a5-9774-b462457dd599 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.069578 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.070224 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.070509 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.070820 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.071548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:26:20.071861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.073444 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.084631 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1403/2810] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:26:20.088638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.098373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e95ee1-0e10-468b-b9a8-1ec5d23ed82a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.103054 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.104121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.104121 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.104501 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.104938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.106852 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.143558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1408/2811] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 01 07:26:20.147903 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.157789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6e508-a271-40ff-a072-ff5f9e068346 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.213685 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.214350 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.214634 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.214944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.215641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.217594 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.237711 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1404/2812] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:20.242389 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.252788 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85e3cd-90d8-40e7-b76b-c7a72f03c851 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.257481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.258137 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.258423 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.258697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.259473 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.261063 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.270848 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1409/2813] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:20.274400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.284256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a23aa0-6a26-4e22-b525-ca48ee6f128b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.289303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.289969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.290253 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.290527 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.291274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.292927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdde261-2b73-4574-9045-4010bf0090b0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.303232 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1405/2814] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:20.307616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.317451 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdee12-a1d2-47bb-a4e8-b218f12144ba admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.321941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.322829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.323118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.323400 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] PATH_INFO: `/v3/projects/5ed0a668502e488abe86e87fa1ef1f33/users/f8a1219cbb794ffeafdc41ef0f4e599a/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.337490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed0a668502e488abe86e87fa1ef1f33, user_id=f8a1219cbb794ffeafdc41ef0f4e599a, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.340175 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.353802 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1410/2815] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:20 2022] PUT /identity/v3/projects/5ed0a668502e488abe86e87fa1ef1f33/users/f8a1219cbb794ffeafdc41ef0f4e599a/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:20.364873 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.365158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.365431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.391476 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] MFA Rules not processed for user `f8a1219cbb794ffeafdc41ef0f4e599a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:20.392747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:20.455972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1406/2816] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:20.461015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.522722 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e7f1-b7a6-4087-a74e-c0f35503d774 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.527215 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.527863 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.528141 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.528415 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.529102 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.531015 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.554796 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1411/2817] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:20.558960 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.570093 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d43a2-25be-4876-8d4d-438b5a8bed27 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.586861 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:20.627231 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.627893 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.628172 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.628447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.629125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.631707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.654789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1407/2818] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:20.657662 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.667804 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672127a-118d-437b-b016-20a522db2b38 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.672293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.672938 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.673218 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.673490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.674227 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.675839 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.686725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1412/2819] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:20.690459 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.701152 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4c00-4a47-4c20-b482-1cb7374089fc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.705629 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.706497 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.706793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.707087 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] PATH_INFO: `/v3/projects/534d5e3e76db496d88572703f10032cf/users/0aae9d2e754a4062972194362f23afb3/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.722609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=534d5e3e76db496d88572703f10032cf, user_id=0aae9d2e754a4062972194362f23afb3, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.725310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.737828 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1408/2820] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:20 2022] PUT /identity/v3/projects/534d5e3e76db496d88572703f10032cf/users/0aae9d2e754a4062972194362f23afb3/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:20.742131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.751970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b03fbe-6f03-4bb5-9cb2-3e3f2ff92dbb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.807560 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.808153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.808436 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.808711 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.809429 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.811051 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5206dc2-1575-45a0-84ee-cc3e3afbe4da admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.821293 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1413/2821] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:20.825269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.835337 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cbef9-f861-467e-ad75-6c652d5a8343 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:20.840105 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:20.840972 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.841255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.841533 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] PATH_INFO: `/v3/projects/534d5e3e76db496d88572703f10032cf/users/0aae9d2e754a4062972194362f23afb3/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.843499 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=534d5e3e76db496d88572703f10032cf, user_id=0aae9d2e754a4062972194362f23afb3, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:20.846114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:20.856035 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1409/2822] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:20 2022] PUT /identity/v3/projects/534d5e3e76db496d88572703f10032cf/users/0aae9d2e754a4062972194362f23afb3/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:20.866235 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:20.866525 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:20.866841 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:20.873407 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:20.893838 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] MFA Rules not processed for user `0aae9d2e754a4062972194362f23afb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:20.895110 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:20.965523 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1414/2823] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:20.977145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:20.978896 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:21.034632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb368cd-1a7b-4964-bec5-555589711fe8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:21.038899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:21.039526 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:21.039808 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:21.040081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:21.048998 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:21.050491 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:21.060841 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1410/2824] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:21.130324 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:21.139976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db6c60-79f1-4c5e-9cd6-5e0d2e0ef5a4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:21.143571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:21.144209 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:21.144490 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:21.144769 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:21.199877 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:21.201307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550ee20-7c63-4789-97d2-b4613a181582 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:21.210136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1415/2825] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:31.056551 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:31.206112 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=60 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:32.392180 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:32.453450 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de00ca-cbc8-4e9a-8b38-bbde7a836b11 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:32.458547 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:32.459430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:32.459719 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:32.459997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] PATH_INFO: `/v3/users/f8a1219cbb794ffeafdc41ef0f4e599a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:32.461179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a1219cbb794ffeafdc41ef0f4e599a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:32.463145 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:32.514360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a1219cbb794ffeafdc41ef0f4e599a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:32.522040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a1219cbb794ffeafdc41ef0f4e599a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:32.522694 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a1219cbb794ffeafdc41ef0f4e599a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:32.534507 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a1219cbb794ffeafdc41ef0f4e599a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:32.535169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a1219cbb794ffeafdc41ef0f4e599a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:32.547044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1411/2826] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:32 2022] DELETE /identity/v3/users/f8a1219cbb794ffeafdc41ef0f4e599a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:32.551570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:32.552058 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:32.552160 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:32.570094 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:32.571341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:32.629951 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1416/2827] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:32.639538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:32.640994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:32.698850 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f605514-d65b-421e-b506-910de5687b69 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:32.702445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:32.703107 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:32.703388 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:32.703662 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:32.712514 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:32.713921 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:32.723148 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1412/2828] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:32.991627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:32.993132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:33.006388 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477e9f5-2e3a-4d5a-9038-801d65600765 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.011000 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.011862 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.012145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.012417 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] PATH_INFO: `/v3/projects/5ed0a668502e488abe86e87fa1ef1f33` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.021071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed0a668502e488abe86e87fa1ef1f33)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.023075 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.068741 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed0a668502e488abe86e87fa1ef1f33', 'request_id': 'req-f9aac4ae-a57c-47b2-8965-ab5c10a90515'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.074739 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed0a668502e488abe86e87fa1ef1f33', 'request_id': 'req-f9aac4ae-a57c-47b2-8965-ab5c10a90515'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.082879 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed0a668502e488abe86e87fa1ef1f33', 'request_id': 'req-f9aac4ae-a57c-47b2-8965-ab5c10a90515'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.092824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] The token cache is being invalidate because project 5ed0a668502e488abe86e87fa1ef1f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:33.093111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.093724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.095477 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1417/2829] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:32 2022] DELETE /identity/v3/projects/5ed0a668502e488abe86e87fa1ef1f33 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:33.100138 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.109699 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb57f-5710-4d43-a4ce-1618517ab01a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.127147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:33.165868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.166725 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.167037 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.167315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] PATH_INFO: `/v3/users/0aae9d2e754a4062972194362f23afb3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.168487 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aae9d2e754a4062972194362f23afb3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.170383 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.216032 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aae9d2e754a4062972194362f23afb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.223317 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aae9d2e754a4062972194362f23afb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.223904 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aae9d2e754a4062972194362f23afb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.235649 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aae9d2e754a4062972194362f23afb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.236290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7c3f1d99-201d-4711-b605-4cc871fb0c2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aae9d2e754a4062972194362f23afb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.248007 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1413/2830] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:33 2022] DELETE /identity/v3/users/0aae9d2e754a4062972194362f23afb3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:33.503855 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.506130 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:33.520540 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aac4ae-a57c-47b2-8965-ab5c10a90515 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.578136 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.579027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.579340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.579612 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] PATH_INFO: `/v3/projects/534d5e3e76db496d88572703f10032cf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.580772 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=534d5e3e76db496d88572703f10032cf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.582700 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.611181 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:33.628751 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534d5e3e76db496d88572703f10032cf', 'request_id': 'req-0f4dbd9e-d6d2-4fbe-a939-a89172460436'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.635197 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534d5e3e76db496d88572703f10032cf', 'request_id': 'req-0f4dbd9e-d6d2-4fbe-a939-a89172460436'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.642264 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534d5e3e76db496d88572703f10032cf', 'request_id': 'req-0f4dbd9e-d6d2-4fbe-a939-a89172460436'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.652464 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] The token cache is being invalidate because project 534d5e3e76db496d88572703f10032cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:33.652752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.653366 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0f4dbd9e-d6d2-4fbe-a939-a89172460436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:33.654902 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1418/2831] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:33 2022] DELETE /identity/v3/projects/534d5e3e76db496d88572703f10032cf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:33.659753 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.660274 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.660386 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.678652 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:33.679880 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:33.737316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1414/2832] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:33.745734 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.746045 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.746323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.767103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:33.768326 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:33.776626 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1419/2833] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:33.780940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.791147 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7d909-1cee-4102-a5e1-6160a59664fc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.795648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.796287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.796568 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.796840 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.797557 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:26:33.797862 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.799470 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.811520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1415/2834] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:26:33.815637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.824752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25861b05-fa4b-4c9d-afa2-a27d1732e935 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.829373 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.830012 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.830296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.830570 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.831285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.833172 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.859404 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1420/2835] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:33 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:33.864255 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.874202 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e4bf9-cc78-4969-af1b-c6f1bd94f48c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.898975 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:33.931411 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.932075 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.932362 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.932638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.933319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.935257 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.954084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1416/2836] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:33 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:33.958106 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:33.968348 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cf89c-f06f-46fa-ab14-83577d49f399 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:33.973481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:33.974157 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:33.974444 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:33.974720 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:33.975478 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:33.977071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:33.987701 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1421/2837] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:33.991916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.001638 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a6a09-f112-4eea-8892-13a41aea1aa3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.006322 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.007004 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.007295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.007573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.008289 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.009981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb8ab6-4116-41cd-9bc2-bcfb32513a62 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.019783 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1417/2838] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:34.023735 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.033526 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1241244-6b64-46df-b224-a78b39e2eb20 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.038663 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.039561 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.039840 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.040113 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] PATH_INFO: `/v3/projects/30454b3616404a7d884ec89ce7d17c07/users/a4142cf89dd64ecdac21643d59030eb3/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.055287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30454b3616404a7d884ec89ce7d17c07, user_id=a4142cf89dd64ecdac21643d59030eb3, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.058230 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.070978 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1422/2839] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:34 2022] PUT /identity/v3/projects/30454b3616404a7d884ec89ce7d17c07/users/a4142cf89dd64ecdac21643d59030eb3/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:26:34.082546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.082843 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.083127 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.110661 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] MFA Rules not processed for user `a4142cf89dd64ecdac21643d59030eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:34.111902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:34.174697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1418/2840] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:34.179081 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.241545 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025ee5d-c79f-4eab-ac85-b64a2a915922 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.246249 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.246941 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.247232 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.247514 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.248195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.250116 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.274746 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1423/2841] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:34 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:34.278793 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.290276 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688957-15b1-4f64-8946-6e5d61540111 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.345752 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.346409 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.346692 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.346997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.348259 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.350188 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.369077 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1419/2842] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:34 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:34.372980 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.383213 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ea082-3bd9-4c10-97ce-45e84f41eb85 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.387834 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.388481 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.388775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.389043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.389760 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.391392 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.402188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1424/2843] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:34.406079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.416919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407400-1a9a-4a99-a08b-7346e35ec1d3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.421717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.422798 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.423088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.423365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] PATH_INFO: `/v3/projects/f735b2a715284a969853ebaf8da3384f/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.438747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f735b2a715284a969853ebaf8da3384f, user_id=3e3e5a3feeaf440c8bfdbd345bdcb7cc, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.441361 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.453962 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1420/2844] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:34 2022] PUT /identity/v3/projects/f735b2a715284a969853ebaf8da3384f/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:34.458341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.468342 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce54c3d-9897-46ef-9835-326b12e79077 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.488338 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:34.527977 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.528638 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.528924 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.529203 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.529921 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.531558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a493-615e-4e71-9a53-be3f4c9bf10b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.541082 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1425/2845] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:26:34.545110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.550479 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:34.557169 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e58a-f0e0-49cc-accd-cf39bcbfd169 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.562120 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.563015 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.563295 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.563569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] PATH_INFO: `/v3/projects/f735b2a715284a969853ebaf8da3384f/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.565535 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f735b2a715284a969853ebaf8da3384f, user_id=3e3e5a3feeaf440c8bfdbd345bdcb7cc, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.568708 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.578543 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1421/2846] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:34 2022] PUT /identity/v3/projects/f735b2a715284a969853ebaf8da3384f/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:34.589184 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.589672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.589790 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.615445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] MFA Rules not processed for user `3e3e5a3feeaf440c8bfdbd345bdcb7cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:34.616673 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:34.686204 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1426/2847] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:26:34.697109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.698900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:34.755119 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c94694-f609-4d45-9f8a-28dc63ecc843 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.758746 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.759403 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.759690 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.759973 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.768906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.770974 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.780329 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1422/2848] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:34.875031 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:34.885219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1699a3-0c14-4b03-b2ea-597ceed5fc07 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:34.888737 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:34.889365 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:34.890140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:34.890140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:34.946149 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:34.947551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e6ef-0eb5-46c1-91ac-be9b7e0881ee service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:34.956314 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1427/2849] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:37.502017 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:37.563483 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9910f-23f0-4f66-8c0d-a9045d0409fc service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:37.567916 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:37.568759 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:37.569040 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:37.569315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] PATH_INFO: `/v3/users/a4142cf89dd64ecdac21643d59030eb3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:37.570498 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4142cf89dd64ecdac21643d59030eb3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:37.572439 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:37.617579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4142cf89dd64ecdac21643d59030eb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:37.624520 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4142cf89dd64ecdac21643d59030eb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:37.625283 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4142cf89dd64ecdac21643d59030eb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:37.636777 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4142cf89dd64ecdac21643d59030eb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:37.637393 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4142cf89dd64ecdac21643d59030eb3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:37.649150 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1423/2850] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:37 2022] DELETE /identity/v3/users/a4142cf89dd64ecdac21643d59030eb3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:37.653630 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:37.654127 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:37.654247 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:37.671830 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:37.673019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:37.711580 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:37.730697 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1428/2851] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:26:37.740795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:37.742205 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:37.757821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:37.812789 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac08b6-a366-4718-8df9-a8136eed8a3e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:37.816445 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:37.817078 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:37.817358 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:37.817632 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:37.826426 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:37.827867 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:37.837086 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1424/2852] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:38.103899 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:38.105354 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:38.120118 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c234d-e30b-4bd1-8536-5051070a6dcd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:38.125143 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:38.126983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.127258 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.127572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] PATH_INFO: `/v3/projects/30454b3616404a7d884ec89ce7d17c07` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.128983 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30454b3616404a7d884ec89ce7d17c07)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:38.131142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:38.177178 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30454b3616404a7d884ec89ce7d17c07', 'request_id': 'req-c1f47bc6-415f-467f-8df9-1c06f1d3d337'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.183439 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30454b3616404a7d884ec89ce7d17c07', 'request_id': 'req-c1f47bc6-415f-467f-8df9-1c06f1d3d337'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.190793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30454b3616404a7d884ec89ce7d17c07', 'request_id': 'req-c1f47bc6-415f-467f-8df9-1c06f1d3d337'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.201533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] The token cache is being invalidate because project 30454b3616404a7d884ec89ce7d17c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:38.201818 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.202442 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.203986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1429/2853] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:38 2022] DELETE /identity/v3/projects/30454b3616404a7d884ec89ce7d17c07 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:38.208447 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:38.218149 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d46e6-0458-49d2-a8f1-ac305cc7b2e5 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:38.274245 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:38.275423 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.275710 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.275983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] PATH_INFO: `/v3/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.277162 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3e5a3feeaf440c8bfdbd345bdcb7cc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:38.279109 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:38.324424 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3e5a3feeaf440c8bfdbd345bdcb7cc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.331281 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3e5a3feeaf440c8bfdbd345bdcb7cc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.332486 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3e5a3feeaf440c8bfdbd345bdcb7cc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.344139 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3e5a3feeaf440c8bfdbd345bdcb7cc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.344767 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7e92f3f-1e46-49f8-995c-7012243227a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3e5a3feeaf440c8bfdbd345bdcb7cc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.356585 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1425/2854] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:38 2022] DELETE /identity/v3/users/3e3e5a3feeaf440c8bfdbd345bdcb7cc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:26:38.619875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:38.621533 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:38.636509 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47bc6-415f-467f-8df9-1c06f1d3d337 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:38.694897 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:38.695750 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.696027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.696296 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] PATH_INFO: `/v3/projects/f735b2a715284a969853ebaf8da3384f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.697449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f735b2a715284a969853ebaf8da3384f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:38.699421 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:38.743211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f735b2a715284a969853ebaf8da3384f', 'request_id': 'req-e84eb9f9-bff8-476a-ac2b-15de9b98c151'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.749145 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f735b2a715284a969853ebaf8da3384f', 'request_id': 'req-e84eb9f9-bff8-476a-ac2b-15de9b98c151'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.756679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f735b2a715284a969853ebaf8da3384f', 'request_id': 'req-e84eb9f9-bff8-476a-ac2b-15de9b98c151'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.763408 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:38.768616 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] The token cache is being invalidate because project f735b2a715284a969853ebaf8da3384f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:38.769103 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.769752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e84eb9f9-bff8-476a-ac2b-15de9b98c151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:38.772092 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1430/2855] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:38 2022] DELETE /identity/v3/projects/f735b2a715284a969853ebaf8da3384f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:38.776715 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.777178 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.777314 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.795072 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:38.796211 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:38.825605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:38.856405 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1426/2856] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:38.915917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.916205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.916482 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.935583 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:38.936895 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:38.945410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1431/2857] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:26:38.949700 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:38.959609 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cd8d5-4ce4-4560-9d41-ea258af184bc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:38.964115 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:38.964757 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:38.965039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:38.965315 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:38.966036 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:26:38.966372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:38.967982 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:38.983666 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1427/2858] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:26:38.988060 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:38.997779 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093cf19-f443-4074-b294-c19041b009a6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.002355 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.003019 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.003300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.003571 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.004243 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.006134 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.030337 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1432/2859] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:38 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:39.034287 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.044641 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f2e48-01b9-4863-8fc8-9bc17b470d38 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.100539 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.101132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.101416 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.101693 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.102378 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.104319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.123837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1428/2860] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:39 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:39.127827 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.138017 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1b44e-58fe-48ed-9d75-c7abd96e0a15 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.142536 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.143206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.143485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.143759 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.144516 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.146096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.156131 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1433/2861] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:39.159762 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.176963 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a824e-4e46-46c8-8004-9a3e34a3a6dc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.182430 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.183114 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.183395 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.183675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.184399 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.185996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9721ba0-cc6f-43bc-aa5a-3aa764483382 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.196095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1429/2862] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 01 07:26:39.200577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.213257 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb4f0a-ae81-4dd4-ad83-e57913bc851f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.218465 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.219394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.219679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.219960 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] PATH_INFO: `/v3/projects/6fea46cbdfb540229f0703ac9804b578/users/86fc93669c014ed2927053f5829600a2/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.235068 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fea46cbdfb540229f0703ac9804b578, user_id=86fc93669c014ed2927053f5829600a2, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.237727 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.250994 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1434/2863] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:39 2022] PUT /identity/v3/projects/6fea46cbdfb540229f0703ac9804b578/users/86fc93669c014ed2927053f5829600a2/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:39.260890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.261184 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.261462 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.287928 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] MFA Rules not processed for user `86fc93669c014ed2927053f5829600a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:39.289123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:39.352687 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1430/2864] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:39.357080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.418874 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3985c-c521-4fc4-8ea2-232731d4bf77 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.423487 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.424132 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.424418 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.424694 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.425381 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.427328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.451752 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1435/2865] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:39 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:39.455668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.466717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622cfe8-f381-425d-a397-d815942f0581 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.492042 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:39.522166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.522837 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.523125 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.523400 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.524079 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.525983 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.545786 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1431/2866] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:39 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:39.549689 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.560332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd6362-7158-4f52-9d44-5a3cc61c08cb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.564976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.565627 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.565914 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.566195 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.566990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.568579 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.578966 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1436/2867] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:39.583373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.593165 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ddb56-2d01-4833-bd35-a6f9283d65a7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.598453 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.599397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.599679 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.599955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] PATH_INFO: `/v3/projects/a2ef3aefd79d45a89d68f09849625fe0/users/f9ce9b058de64fc1ac56e0c7dd139af8/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.614397 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ef3aefd79d45a89d68f09849625fe0, user_id=f9ce9b058de64fc1ac56e0c7dd139af8, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.617534 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.629868 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1432/2868] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:39 2022] PUT /identity/v3/projects/a2ef3aefd79d45a89d68f09849625fe0/users/f9ce9b058de64fc1ac56e0c7dd139af8/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 01 07:26:39.633832 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.643569 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73aa0a-83e2-4ef7-9529-affc6580c948 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.697764 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.698410 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.698690 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.698996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.699707 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.701285 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6995f7-aaf7-46f9-9e7e-cde58c36b563 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.711270 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1437/2869] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:39.715273 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.725993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf4046-5bb6-4887-8fe6-faa615fb5566 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.730559 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.731463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.731750 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.732029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] PATH_INFO: `/v3/projects/a2ef3aefd79d45a89d68f09849625fe0/users/f9ce9b058de64fc1ac56e0c7dd139af8/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.733997 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ef3aefd79d45a89d68f09849625fe0, user_id=f9ce9b058de64fc1ac56e0c7dd139af8, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.736640 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.746643 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1433/2870] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:39 2022] PUT /identity/v3/projects/a2ef3aefd79d45a89d68f09849625fe0/users/f9ce9b058de64fc1ac56e0c7dd139af8/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:39.756891 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.757188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.757470 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.764027 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:39.784265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] MFA Rules not processed for user `f9ce9b058de64fc1ac56e0c7dd139af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:39.785471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:39.850558 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1438/2871] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:39.861058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:39.862801 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:39.919018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f25125-de31-4260-b477-b8a7591dc03b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:39.922573 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:39.923230 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:39.923518 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:39.923797 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:39.932943 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:39.934373 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04acea2-4636-4ca6-b62a-042894d1782b service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:39.944500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1434/2872] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:40.290919 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:40.350821 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feace8b2-bacc-4791-acaf-3600f5f5da98 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:40.355240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:40.356093 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:40.356378 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:40.356653 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] PATH_INFO: `/v3/users/86fc93669c014ed2927053f5829600a2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:40.357817 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fc93669c014ed2927053f5829600a2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:40.359757 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:40.403745 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fc93669c014ed2927053f5829600a2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.410637 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fc93669c014ed2927053f5829600a2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.411265 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fc93669c014ed2927053f5829600a2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.422906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fc93669c014ed2927053f5829600a2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.423474 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fc93669c014ed2927053f5829600a2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.436105 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1439/2873] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:40 2022] DELETE /identity/v3/users/86fc93669c014ed2927053f5829600a2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:40.440548 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:40.440887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:40.441164 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:40.458914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:40.460101 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:40.517159 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1435/2874] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:40.526755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:40.528202 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:40.539394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:40.590282 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696028cc-964e-4055-919e-701f76332b26 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:40.593956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:40.594605 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:40.594930 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:40.595211 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:40.604655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:40.606087 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:40.616098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1440/2875] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:26:40.875372 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:40.876795 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:40.889675 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e330ae-7211-43da-bc48-0e1c3c02ee5c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:40.894996 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:40.895906 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:40.896256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:40.896598 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] PATH_INFO: `/v3/projects/6fea46cbdfb540229f0703ac9804b578` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:40.897899 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fea46cbdfb540229f0703ac9804b578)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:40.900179 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:40.922969 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:40.952000 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fea46cbdfb540229f0703ac9804b578', 'request_id': 'req-4084fb3d-ec97-4067-b15b-31f2287eddce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.959770 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fea46cbdfb540229f0703ac9804b578', 'request_id': 'req-4084fb3d-ec97-4067-b15b-31f2287eddce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.966095 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fea46cbdfb540229f0703ac9804b578', 'request_id': 'req-4084fb3d-ec97-4067-b15b-31f2287eddce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.977260 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] The token cache is being invalidate because project 6fea46cbdfb540229f0703ac9804b578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:40.977555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.978204 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:40.979811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1436/2876] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:40 2022] DELETE /identity/v3/projects/6fea46cbdfb540229f0703ac9804b578 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:40.983758 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:40.993588 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5529-dada-44b7-8aa3-999716e842fd service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.052188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.053050 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.053331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.053607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] PATH_INFO: `/v3/users/f9ce9b058de64fc1ac56e0c7dd139af8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.054780 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ce9b058de64fc1ac56e0c7dd139af8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.056948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.102869 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ce9b058de64fc1ac56e0c7dd139af8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.109939 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ce9b058de64fc1ac56e0c7dd139af8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.110551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ce9b058de64fc1ac56e0c7dd139af8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.122193 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ce9b058de64fc1ac56e0c7dd139af8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.122851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-352dd8d1-5236-4477-baa3-464b1f72fb56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ce9b058de64fc1ac56e0c7dd139af8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.134631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1441/2877] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:26:40 2022] DELETE /identity/v3/users/f9ce9b058de64fc1ac56e0c7dd139af8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:41.398844 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.400446 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:41.415847 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084fb3d-ec97-4067-b15b-31f2287eddce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.474262 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.475096 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.475380 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.475655 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] PATH_INFO: `/v3/projects/a2ef3aefd79d45a89d68f09849625fe0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.476859 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ef3aefd79d45a89d68f09849625fe0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.478788 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.523731 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ef3aefd79d45a89d68f09849625fe0', 'request_id': 'req-19138c9d-0d48-43bc-a72b-dd89edde03e1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.530624 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ef3aefd79d45a89d68f09849625fe0', 'request_id': 'req-19138c9d-0d48-43bc-a72b-dd89edde03e1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.536894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ef3aefd79d45a89d68f09849625fe0', 'request_id': 'req-19138c9d-0d48-43bc-a72b-dd89edde03e1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.547601 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] The token cache is being invalidate because project a2ef3aefd79d45a89d68f09849625fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:26:41.547887 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.548538 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-19138c9d-0d48-43bc-a72b-dd89edde03e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:26:41.550084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1437/2878] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:26:41 2022] DELETE /identity/v3/projects/a2ef3aefd79d45a89d68f09849625fe0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:41.556551 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.556837 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.557114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.575433 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:41.576618 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:41.601692 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:41.635754 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1442/2879] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 01 07:26:41.645272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.645272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.645555 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.667900 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:41.669124 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:41.681754 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1438/2880] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:41.682755 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.693530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b7f6a-bbe8-40e9-beb2-51e61c9283f0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.698240 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.698929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.699267 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.699550 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.700274 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 01 07:26:41.700592 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.702216 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.714686 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1443/2881] 10.222.0.48 () {58 vars in 1152 bytes} [Fri Jul 1 07:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 01 07:26:41.719187 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.729157 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e778b-6bf5-4b1b-8ecb-7cc3cc2a8715 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.733944 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.734648 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.734981 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.735269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.735959 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.737855 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.767811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1439/2882] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:41 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:41.772120 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.785281 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505e2b-99a3-440e-9452-b0ab46f1aa44 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.846986 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.847738 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.848023 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.848300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.848987 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.851785 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.873691 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1444/2883] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:41 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:41.877948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.889305 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dea26a-6720-4a96-abec-583838c6c2c9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.893914 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.894605 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.894917 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.895197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.895915 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.897500 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.899561 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:41.909467 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1440/2884] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:41.913014 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.923631 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cadd8-d35b-4c15-ba4e-14e2da81523c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.928876 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.929537 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.929820 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.930096 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.930853 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.932705 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7993-477b-4477-8ad8-37c51f7c2e42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.943150 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1445/2885] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:41.947033 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:41.956986 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebb1f-bfd8-4299-a5bb-94cd1bfc9296 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:41.961627 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:41.962492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:41.962811 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:41.963084 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] PATH_INFO: `/v3/projects/f12c0a38c96d474e98388f240d002d71/users/3c6974def201473f9a8398e300cdd6ea/roles/41985665942d4bba9f8355bf503f068b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:41.979460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12c0a38c96d474e98388f240d002d71, user_id=3c6974def201473f9a8398e300cdd6ea, role_id=41985665942d4bba9f8355bf503f068b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:41.982061 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:41.995398 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1441/2886] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:41 2022] PUT /identity/v3/projects/f12c0a38c96d474e98388f240d002d71/users/3c6974def201473f9a8398e300cdd6ea/roles/41985665942d4bba9f8355bf503f068b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:42.000512 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376dbbd-4c4c-470e-9a41-ca19d2429d79 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:42.001003 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376dbbd-4c4c-470e-9a41-ca19d2429d79 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:42.001111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376dbbd-4c4c-470e-9a41-ca19d2429d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:42.018263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6376dbbd-4c4c-470e-9a41-ca19d2429d79 None None] MFA Rules not processed for user `bb77d87b954348508ac808b62617ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:42.019517 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6376dbbd-4c4c-470e-9a41-ca19d2429d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:42.076716 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1446/2887] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:42.086053 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:42.087728 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:42.146296 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316a3ec-9877-40b9-b4e5-61316f930af2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:42.149883 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:42.150510 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:42.150809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:42.151090 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:42.159809 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:42.161740 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:42.170923 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1442/2888] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:26:45.925154 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:45.925731 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:45.926098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:45.956251 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] MFA Rules not processed for user `3c6974def201473f9a8398e300cdd6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:45.960810 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:45.990577 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:46.040347 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1447/2889] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:46.045009 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.055319 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11106-30c1-4c40-b834-7c021088e0e8 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.060494 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.061158 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.061441 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.061717 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.062414 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.064391 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.096987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1443/2890] 10.222.0.48 () {60 vars in 1152 bytes} [Fri Jul 1 07:26:46 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:46.099300 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.111340 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b33613-6b7a-4660-a933-5b641c38bba6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.173575 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.174362 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.174652 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.175303 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.176613 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.178674 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.207679 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1448/2891] 10.222.0.48 () {60 vars in 1143 bytes} [Fri Jul 1 07:26:46 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 01 07:26:46.209039 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.223554 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601998-3065-4aad-9990-b4c9cd7ad2ea admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.229026 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.230137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.230137 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.230365 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.231239 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.232905 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.244964 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1444/2892] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:46.249047 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.259976 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b246729-d19f-47cb-99ef-7c0d83de3b67 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.268782 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.271205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.271205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.271205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] PATH_INFO: `/v3/projects/840583cb484442a39aaba89000bb347c/users/e930a2408d924a4fba469e620c62d057/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.286352 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840583cb484442a39aaba89000bb347c, user_id=e930a2408d924a4fba469e620c62d057, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.289763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.306305 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1449/2893] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:46 2022] PUT /identity/v3/projects/840583cb484442a39aaba89000bb347c/users/e930a2408d924a4fba469e620c62d057/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:46.310611 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.322586 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972da24-b569-43bf-ae3f-919618d1b8ee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.383049 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.383727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.384016 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.384297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.385029 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.386635 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd31551-62d5-4a8a-9ad7-9129baf0f4d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.397271 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1445/2894] 10.222.0.48 () {58 vars in 1121 bytes} [Fri Jul 1 07:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 01 07:26:46.401367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:46.412426 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb20850d-a39b-4520-b0c0-54e355c343a3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:46.417177 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:46.418059 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:46.418341 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:46.418617 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] PATH_INFO: `/v3/projects/840583cb484442a39aaba89000bb347c/users/e930a2408d924a4fba469e620c62d057/roles/c003a22b06be43a186fa3414e2b93df2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:46.420655 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] RBAC: Authorizing `identity:create_grant(project_id=840583cb484442a39aaba89000bb347c, user_id=e930a2408d924a4fba469e620c62d057, role_id=c003a22b06be43a186fa3414e2b93df2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:46.423331 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:46.434253 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1450/2895] 10.222.0.48 () {60 vars in 1482 bytes} [Fri Jul 1 07:26:46 2022] PUT /identity/v3/projects/840583cb484442a39aaba89000bb347c/users/e930a2408d924a4fba469e620c62d057/roles/c003a22b06be43a186fa3414e2b93df2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:26:50.886987 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b11cc-9243-47a0-8808-62bad3a9a20b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:50.887474 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b11cc-9243-47a0-8808-62bad3a9a20b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:50.887593 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b11cc-9243-47a0-8808-62bad3a9a20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:50.919429 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c1b11cc-9243-47a0-8808-62bad3a9a20b None None] MFA Rules not processed for user `e930a2408d924a4fba469e620c62d057`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:26:50.920660 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c1b11cc-9243-47a0-8808-62bad3a9a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:50.969030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:51.022698 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1446/2896] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:26:51.044034 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:26:51.048124 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:26:51.157480 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c76a8-3e0e-4533-84fa-5427d98c428e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:26:51.169080 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:26:51.169915 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:26:51.170205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:26:51.170485 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:26:51.195049 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:26:51.244672 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:26:51.246280 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:26:51.257044 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1451/2897] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:26:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:27:01.018489 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=16 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:01.251538 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=20 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:12.688374 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:27:12.688870 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:27:12.688985 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:27:12.706068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] MFA Rules not processed for user `3c6974def201473f9a8398e300cdd6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:27:12.707304 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:12.714146 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1447/2898] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:27:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:27:13.460278 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:27:13.461742 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:13.533908 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb7c0d-0d5d-4fe8-8f94-395efafd3135 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:27:13.538970 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:27:13.539615 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:27:13.539900 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:27:13.540180 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:27:13.541950 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:13.553153 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:27:13.554730 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:27:13.564771 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1452/2899] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:27:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:27:22.700927 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:23.560646 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:27.360902 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:27:27.377835 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac7ab-ccac-488a-95f9-841cdb022438 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:27:27.384014 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:27:27.384667 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:27:27.384947 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:27:27.385220 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:27:27.387094 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:27.402038 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:27:27.404132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:27:27.417297 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1448/2900] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:27:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:27:30.023114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:27:30.024528 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:30.078829 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd988f-417d-4054-962f-a1e487cfe0f1 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:27:30.082367 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:27:30.083642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:27:30.083927 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:27:30.084205 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:27:30.093029 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:27:30.094446 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:27:30.104107 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1453/2901] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:27:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:27:37.412550 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:38.124406 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:27:38.136256 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef145d-c73a-4dc4-9205-f980e82a3238 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:27:38.141123 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:27:38.141747 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:27:38.142028 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:27:38.142303 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:27:38.144058 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:27:38.155196 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:27:38.156738 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:27:38.166898 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1449/2902] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:27:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:27:40.099861 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=26 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:27:48.162706 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:29:39.812238 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:29:39.820870 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb163-d30a-49e4-b618-42b33641c0fb service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:29:39.824471 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:29:39.825098 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:29:39.825377 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:29:39.825650 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:29:39.827419 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:29:39.836670 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:29:39.838176 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:29:39.846608 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1454/2903] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:29:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:29:45.732041 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:29:45.741574 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c2b26-b652-474f-83a1-d62d3471dd03 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:29:45.745140 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:29:45.745771 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:29:45.746055 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:29:45.746331 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:29:45.748118 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:29:45.761838 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:29:45.763290 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e565d-0be3-4f74-a7e2-524490dfc70f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:29:45.774702 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1450/2904] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:29:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:29:46.949151 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:29:46.958438 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8ce67-fe02-4269-889b-778a988ed2a4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:29:46.962089 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:29:46.962725 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:29:46.963043 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:29:46.963323 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:29:46.965085 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:29:46.979302 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:29:46.980724 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:29:46.993114 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1455/2905] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:29:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:29:49.270394 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:29:49.270958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:29:49.270958 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:29:49.289043 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] MFA Rules not processed for user `3c6974def201473f9a8398e300cdd6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:29:49.290264 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:29:49.297336 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1451/2906] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:29:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:29:49.602906 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:29:49.611530 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc12967-5649-4380-b1b2-056a12672a6f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:29:49.615219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:29:49.615851 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:29:49.616133 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:29:49.616409 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:29:49.618140 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:29:49.628156 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:29:49.629572 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c2cc-31bf-4b07-a123-a8e8dbd3a9f2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:29:49.637505 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1456/2907] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:29:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 01 07:29:59.283569 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=9 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:29:59.633917 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=18 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:32:01.281197 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:32:01.282590 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:32:01.338712 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9cd5-a85f-4b2c-9758-2d3c2229c327 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:32:01.342360 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52d7bcc4b67c495f821519996cc2ab26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:32:01.343018 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:01.343302 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:01.343579 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:01.352143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:32:01.353563 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:32:01.362166 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1452/2908] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:32:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:32:11.358272 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=26 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:32:12.318332 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:12.318873 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:12.318990 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:12.337460 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] MFA Rules not processed for user `3c6974def201473f9a8398e300cdd6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 01 07:32:12.338642 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:32:12.345370 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1457/2909] 10.222.0.48 () {58 vars in 957 bytes} [Fri Jul 1 07:32:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 01 07:32:12.663193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:32:12.672821 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4ec08-283c-49f2-9667-7456a7e0c6d2 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:32:12.677154 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:32:12.677993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:12.678431 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:12.678890 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:12.680994 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:32:12.690940 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:32:12.692875 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:32:12.700955 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1453/2910] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:32:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:32:15.835702 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:32:15.837152 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:32:15.858287 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0faa3cd-5fbd-4724-a9e0-2a4d7e51cdb2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:32:15.864022 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:32:15.864677 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:15.864962 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:15.865368 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:15.867307 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:32:15.880775 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:32:15.882437 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:32:15.894394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1458/2911] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:32:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:32:22.697193 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:32:25.889263 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=9 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:32:29.883546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:32:29.892081 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da554e-58aa-48fb-9e5a-02828f7fea39 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:32:29.896310 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:32:29.896931 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:29.897218 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:29.897492 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:29.906269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:32:29.907718 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:32:29.917088 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1454/2912] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:32:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:32:39.912907 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:32:40.078826 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:32:40.135084 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113c3d2-2bcf-48e0-8be1-f2b6a1da13f2 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:32:40.140188 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:32:40.140824 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:32:40.141111 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:32:40.141390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:32:40.150445 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:32:40.152071 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:32:40.162502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1459/2913] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:32:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:32:50.157687 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:34:33.714285 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:33.783707 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a8ce-b9c7-40f6-a015-1f21e4b4cfe9 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:33.790697 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '458dc4b52e3d4a13a8a4f801ddf2e6c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:33.791363 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:33.791646 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:33.791919 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:33.793677 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:34:33.839418 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:33.841307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:33.856376 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1455/2914] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:34:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:34:40.468586 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:40.496956 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170485b-04c1-4312-a9ec-982d0cc03ce3 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:40.501607 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:40.502845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:40.502845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:40.502845 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:40.515038 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:40.516793 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:40.527682 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1460/2915] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:34:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:34:43.850894 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=36 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:34:46.237183 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:46.247313 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d84497-0087-4bbb-a0ad-9ed728577030 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:46.250874 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58947ba37ab14f61adde0bf2f34829f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:46.251512 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:46.251807 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:46.252083 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:46.312948 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:46.314307 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:46.323941 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1456/2916] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:34:49.299401 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:49.300819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:34:49.329219 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0ceaf-598a-4dab-9338-c7e94f917c17 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:49.334250 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:49.334929 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:49.335214 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:49.336091 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:49.337875 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:34:49.353062 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:49.354996 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:49.378268 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1461/2917] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:34:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:34:52.264949 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:52.281657 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b47caa-15e5-470c-993b-59720feff39a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:52.282316 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:52.283261 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:52.283549 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:52.283826 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] PATH_INFO: `/v3/users/3c6974def201473f9a8398e300cdd6ea` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:52.285030 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6974def201473f9a8398e300cdd6ea)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:52.287288 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:52.333160 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6974def201473f9a8398e300cdd6ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.341417 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6974def201473f9a8398e300cdd6ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.342068 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6974def201473f9a8398e300cdd6ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.355463 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6974def201473f9a8398e300cdd6ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.356163 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6974def201473f9a8398e300cdd6ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.369849 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1457/2918] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:34:52 2022] DELETE /identity/v3/users/3c6974def201473f9a8398e300cdd6ea => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:34:52.549598 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:52.565948 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167acae-8c47-48f6-8cd9-4b029a5014c7 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:52.622809 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:52.623669 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:52.623952 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:52.624226 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] PATH_INFO: `/v3/projects/f12c0a38c96d474e98388f240d002d71` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:52.625390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12c0a38c96d474e98388f240d002d71)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:52.627327 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:52.672208 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12c0a38c96d474e98388f240d002d71', 'request_id': 'req-6660d074-e67b-4276-b10c-c13f44fa4df2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.678142 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12c0a38c96d474e98388f240d002d71', 'request_id': 'req-6660d074-e67b-4276-b10c-c13f44fa4df2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.685449 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12c0a38c96d474e98388f240d002d71', 'request_id': 'req-6660d074-e67b-4276-b10c-c13f44fa4df2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.695910 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] The token cache is being invalidate because project f12c0a38c96d474e98388f240d002d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:34:52.696199 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.696819 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.698328 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1462/2919] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:34:52 2022] DELETE /identity/v3/projects/f12c0a38c96d474e98388f240d002d71 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:34:52.702460 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:52.703897 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:34:52.717381 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f812740-436f-4982-b4f7-3f346b387f16 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:52.774186 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:52.775110 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:52.775390 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:52.775668 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] PATH_INFO: `/v3/users/e930a2408d924a4fba469e620c62d057` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:52.776865 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e930a2408d924a4fba469e620c62d057)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:52.778993 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:52.784143 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:34:52.825425 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e930a2408d924a4fba469e620c62d057'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.832066 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e930a2408d924a4fba469e620c62d057'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.832848 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e930a2408d924a4fba469e620c62d057'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.844461 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e930a2408d924a4fba469e620c62d057'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.845132 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e930a2408d924a4fba469e620c62d057'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:52.857044 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1458/2920] 10.222.0.48 () {58 vars in 1223 bytes} [Fri Jul 1 07:34:52 2022] DELETE /identity/v3/users/e930a2408d924a4fba469e620c62d057 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:34:53.027502 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:34:53.028934 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:34:53.044390 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660d074-e67b-4276-b10c-c13f44fa4df2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:34:53.084448 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:34:53.103685 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb77d87b954348508ac808b62617ddb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0142643ea3d4627a8dc9edbdfab9d31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:34:53.104541 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:34:53.104825 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:34:53.105101 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] PATH_INFO: `/v3/projects/840583cb484442a39aaba89000bb347c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:34:53.106275 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840583cb484442a39aaba89000bb347c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:34:53.108206 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:34:53.152521 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840583cb484442a39aaba89000bb347c', 'request_id': 'req-415b2fce-ac3d-445b-8375-8ef2e0f5da84'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:53.158394 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840583cb484442a39aaba89000bb347c', 'request_id': 'req-415b2fce-ac3d-445b-8375-8ef2e0f5da84'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:53.166259 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840583cb484442a39aaba89000bb347c', 'request_id': 'req-415b2fce-ac3d-445b-8375-8ef2e0f5da84'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:53.176680 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] The token cache is being invalidate because project 840583cb484442a39aaba89000bb347c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 01 07:34:53.176964 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:53.177581 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-415b2fce-ac3d-445b-8375-8ef2e0f5da84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 01 07:34:53.180018 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1463/2921] 10.222.0.48 () {58 vars in 1232 bytes} [Fri Jul 1 07:34:53 2022] DELETE /identity/v3/projects/840583cb484442a39aaba89000bb347c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 01 07:35:02.853567 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:35:03.173763 dv-cinder-807215-2-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 01 07:37:45.074727 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 01 07:37:45.076323 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:37:45.147619 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773b8e3-4274-42e8-88d6-914b5164f439 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 01 07:37:45.211338 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41b5461ab9d746b2b45cbc1960dececd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '269f65c9d92249e5a62bbf25c237b2a8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 01 07:37:45.211984 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 01 07:37:45.212269 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 01 07:37:45.212544 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 01 07:37:45.214332 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 01 07:37:45.225546 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 01 07:37:45.227151 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1167c18-1d1e-4ec5-ae67-b1fb019e9295 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 01 07:37:45.236885 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1459/2922] 10.222.0.48 () {60 vars in 1365 bytes} [Fri Jul 1 07:37:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 01 07:37:55.232612 dv-cinder-807215-2-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=58 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}